I’ve always found it hilarious to think that LLMs to date have a habit of returning delusional answers rather than simply state they “can’t compute”. It also explains in an oddly mirrored way why some human managers I’ve had the displeasure of working with or for make some totally stupid and irrational decisions! Lol.
As with any system, it can go wrong and it’s important to engineer in ways of at least detecting and reporting on strange behaviour as soon as it is detected by investing in defensive coding. Not sure what the AI equivalent is but humans spend many years learning various rules or guidelines around morality and being honest and admit your limits or failures, though even that has been severely eroded in todays selfish and self promoting world - spot the grumpy old man about to graduate from his fifties!
As such, I think testing is the way to go and it shouldn’t just stop at release date. I am a big proponent of live testing and passing through (be it specially marked) test requests into systems to see how they fare.
In the current world of MVP madness though, all of those features get deprioritised and end up on the backlog and entries in the technical debt and deep in the in-tray of the product manager.
Good luck with your crusade, and if I can help, you have my full support.
Great article, Devonshire. At our startup, we are living forever on the cusp of the perfect release because of just that kind of short-sightedness with regard to flaws in system design. "Yes, Development, it absolutely does work correctly if the user does everything right, the way we intend. But look here, see how badly it collapses when they don't".
I will share at the office.
(You had a little typo that reversed the intent of the Tyson quote - its "Everyone's got a *plan*...)
I’ve always found it hilarious to think that LLMs to date have a habit of returning delusional answers rather than simply state they “can’t compute”. It also explains in an oddly mirrored way why some human managers I’ve had the displeasure of working with or for make some totally stupid and irrational decisions! Lol.
As with any system, it can go wrong and it’s important to engineer in ways of at least detecting and reporting on strange behaviour as soon as it is detected by investing in defensive coding. Not sure what the AI equivalent is but humans spend many years learning various rules or guidelines around morality and being honest and admit your limits or failures, though even that has been severely eroded in todays selfish and self promoting world - spot the grumpy old man about to graduate from his fifties!
As such, I think testing is the way to go and it shouldn’t just stop at release date. I am a big proponent of live testing and passing through (be it specially marked) test requests into systems to see how they fare.
In the current world of MVP madness though, all of those features get deprioritised and end up on the backlog and entries in the technical debt and deep in the in-tray of the product manager.
Good luck with your crusade, and if I can help, you have my full support.
Thank you. An easy way to show support would be to share my work with people on various platforms. That helps me grow and enables new readers.
You might also consider getting a premium subscription to Tech Made Simple to support my writing
Great article, Devonshire. At our startup, we are living forever on the cusp of the perfect release because of just that kind of short-sightedness with regard to flaws in system design. "Yes, Development, it absolutely does work correctly if the user does everything right, the way we intend. But look here, see how badly it collapses when they don't".
I will share at the office.
(You had a little typo that reversed the intent of the Tyson quote - its "Everyone's got a *plan*...)
Good catch. No worries about the name
LOL speaking of typos...
Sorry, Devansh, I didn't catch the Spell-wreck correction of your name until after I posted