Interesting Content in AI, Software, Business, and Tech- 11/1/2023 [Updates]
Content to help you keep up with Machine Learning, Deep Learning, Data Science, Software Engineering, Finance, Business, and more
Hey, it’s Devansh 👋👋
In issues of Updates, I will share interesting content I came across. While the focus will be on AI and Tech, the ideas might range from business, philosophy, ethics, and much more. The goal is to share interesting content with y’all so that you can get a peek behind the scenes into my research process.
If you’d like to support my writing, consider becoming a premium subscriber to my sister publication Tech Made Simple to support my crippling chocolate milk addiction. Use the button below for a discount.
p.s. you can learn more about the paid plan here. If you’re looking for AI Consulting, feel free to reach out.
A lot of people reach out to me for reading recommendations. I figured I’d start sharing whatever AI Papers/Publications, interesting books, videos, etc I came across each week. Some will be technical, others not really. I will add whatever content I found really informative (and I remembered throughout the week). These won’t always be the most recent publications- just the ones I’m paying attention to this week. Without further ado, here are interesting readings/viewings for 11/1/2023. If you missed last week’s readings, you can find it here.
Reminder- We started an AI Made Simple Subreddit. Come join us over here- https://www.reddit.com/r/AIMadeSimple/
Community Spotlight- Adam Haney
Adam Haney is an engineering leader, investor, and overall cool guy. He is currently the VP of Engineering at Invisible Technologies, a company that has streamlined data operations for various Leaders- including Microsoft, Cohere, Nasdaq, and Doordash. We had a conversation earlier this week, and he expressed interest in networking with more people in the space. If you're interested in having a conversation with him, please reach out to him.
If you're doing interesting work and would like to be featured in the spotlight section, just drop your introduction in the comments/by reaching out to me. There are no rules- you could talk about a paper you've written, an interesting project you've worked on, some personal challenge you're working on, ask me to promote your company/product, or anything else you consider important. The goal is to get to know you better, and possibly connect you with interesting people in our chocolate milk cult. No costs/obligations are attached.
Also, I will be moving to NYC before the end of this month. I’m considering hosting regular meetups to get to know people/connect people with each other. Is that something that interests you?
Highly Recommended
These are pieces that I feel are particularly well done. If you don't have much time, make sure you at least catch these works.
Are AI products really making us more productive?!
A simple post by Damien Benveniste, PhD hits the problem with the AI Tools industry right on the head.
I often see people on LinkedIn posting about the "10 AI products that will make you more productive". Well, I use some of those products on a daily basis, and I really wish that AI could take a back seat to what users actually need!And I feel there is this trend for many companies! This desire to release a new AI feature cannibalizes the development of the core features that are actually useful. AI became a marketing tool that leads to useless products or features just for the sake of pretending to be technologically up-to-date in this AI hype! Hope this changes soon!
Why is CatBoost better then XGBoost and LightGBM.
Valeriy Manokhin, PhD, MBA, CQF has some great insights on Time Series predictions and Tabular ML. He had a very interesting post talking about why CatBoost outperforms-
CatBoost has better generalisation capabilities due to fundamentally better and much more clever design. By utilising ideas such as Ordered Target Statistics from online learning, CatBoost considers datasets sequential in time.It then permutes them to reduce potential leakage called Prediction Shift, inherent in the traditional Gradient Boosting models such as XGBoost and LightGBM.By creating the concept of artificial time 🕰️ CatBoost cleverly reduces Prediction Shift.
Cognitive Bias at the Coffee Shop
I've long maintained that studying cognitive biases is a must for any Data Scientist. It is one of the most humbling experiences. Our main man Andrew dropped a gem with his piece covering some important biases. I'm honestly jealous of his ability to craft fictional scenarios to illustrate his point.
Despite observing signs that the clerk might not be a morning person (like her yawning), you selectively focused on her efficiency, which confirms your initial belief that she's a morning person.
This is a classic example of confirmation bias. It's our tendency to seek, favor, and remember information that confirms our pre-existing beliefs, while often ignoring or dismissing information that contradicts them. Ever hear of the phrase “selective hearing” or “cherry picking”? Think about those phrases when you hear “confirmation bias.”
The Germans: Schopenhauer
I've been binge watching Wes Cecil's YouTube channel recently. He does very interesting videos on different philosophers. I'm linking his lecture on Schopenhauer because in the end- Cecil talks about how important it is to have alternative streams of thought/success in a field. I think it's a statement worth tattooing on your forehead (and something that AI and Academia is failing at this moment). Open Source has been exceptional because it gives space to everyone, and we need to encourage that more.
FOD#26: AI Hypocrisy
Ksenia Se is one of a kind when it comes to blending AI News with Insightful Analysis on Societal/Business Impacts. This one is a perfect demonstration of where companies are putting their money, and how it conflicts with the popular narrative.
The other conclusion is that the main danger(s) is overlooked: nuclear war. It feels like we are currently in the same situation. We focus on fictional fears and overlook the real dangers present.
Our AI systems offer us convenience, yet we're preoccupied with the idea of them exterminating us. Why and how did we decide an algorithm's objective would be to wipe us out? What you heard in Terminator is simply untrue. The whole premise about machines wanting to kill us is wrong.
JD's Brief Notes on Recent Noteworthy Papers
Jean David Ruvini has had enough of me. Not really (or so I hope), but if he continues to do such fantastic paper notes, I'm going to be out of a job very soon. Go show JDR some encouragement, his summaries are really good (especially for someone just starting).
Sharing my notes about some papers I've found interesting lately. My notes may be a bit cryptic; I hope they're still useful in piquing your curiosity and helping you breathing under the waterfall.
Is This A Golden Age of Fraud?
Patrick Boyle combines deep analysis with his trademark dry humor to drop another instant classic. My favorite part- "The man who wrote the 4 hour work week spent more than way 4 hours a week promoting his book."
A video about how "passive income" money-making scams seem to have taken over the world, and the economic implications of such scams.
How to Beat Signal Jamming with Low Resources
In preparation, for the upcoming post on beating government surveillance and internet suppression, I put together this piece for the times where signal jamming is deployed to cause communication blackouts.
A common tactic used to disrupt civilian protests and control the narrative is signal jamming. By employing large scale signal jamming, one can disrupt communications and cause blackouts- allowing them to paint the narrative however they please. Signal Jamming also disrupts communication , making large scale coordination difficult. Fortunately there are ways to fight back. By analyzing the signals received during jamming, we can start to infer properties of the interfering jammers. This post will explain the basic techniques you need to develop your own lightweight anti-signal jamming tools. We will use AI to analyze the properties of the disrupted signal, and in doing so be able to work around the blockage.
AI Content
The Devil is in the GAN: Backdoor Attacks and Defenses in Deep Generative Models
Deep Generative Models (DGMs) are a popular class of deep learning models which find widespread use because of their ability to synthesize data from complex, high-dimensional manifolds. However, even with their increasing industrial adoption, they haven't been subject to rigorous security and privacy analysis. In this work we examine one such aspect, namely backdoor attacks on DGMs which can significantly limit the applicability of pre-trained models within a model supply chain and at the very least cause massive reputation damage for companies outsourcing DGMs form third parties.
While similar attacks scenarios have been studied in the context of classical prediction models, their manifestation in DGMs hasn't received the same attention. To this end we propose novel training-time attacks which result in corrupted DGMs that synthesize regular data under normal operations and designated target outputs for inputs sampled from a trigger distribution. These attacks are based on an adversarial loss function that combines the dual objectives of attack stealth and fidelity. We systematically analyze these attacks, and show their effectiveness for a variety of approaches like Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs), as well as different data domains including images and audio. Our experiments show that - even for large-scale industry-grade DGMs (like StyleGAN) - our attacks can be mounted with only modest computational effort. We also motivate suitable defenses based on static/dynamic model and output inspections, demonstrate their usefulness, and prescribe a practical and comprehensive defense strategy that paves the way for safe usage of DGMs.
Subtle adversarial image manipulations influence both human and machine perception
Although artificial neural networks (ANNs) were inspired by the brain, ANNs exhibit a brittleness not generally observed in human perception. One shortcoming of ANNs is their susceptibility to adversarial perturbations—subtle modulations of natural images that result in changes to classification decisions, such as confidently mislabelling an image of an elephant, initially classified correctly, as a clock. In contrast, a human observer might well dismiss the perturbations as an innocuous imaging artifact. This phenomenon may point to a fundamental difference between human and machine perception, but it drives one to ask whether human sensitivity to adversarial perturbations might be revealed with appropriate behavioral measures. Here, we find that adversarial perturbations that fool ANNs similarly bias human choice. We further show that the effect is more likely driven by higher-order statistics of natural images to which both humans and ANNs are sensitive, rather than by the detailed architecture of the ANN.
Patch-Fool: Are Vision Transformers Always Robust Against Adversarial Perturbations?
Vision transformers (ViTs) have recently set off a new wave in neural architecture design thanks to their record-breaking performance in various vision tasks. In parallel, to fulfill the goal of deploying ViTs into real-world vision applications, their robustness against potential malicious attacks has gained increasing attention. In particular, recent works show that ViTs are more robust against adversarial attacks as compared with convolutional neural networks (CNNs), and conjecture that this is because ViTs focus more on capturing global interactions among different input/feature patches, leading to their improved robustness to local perturbations imposed by adversarial attacks. In this work, we ask an intriguing question: "Under what kinds of perturbations do ViTs become more vulnerable learners compared to CNNs?" Driven by this question, we first conduct a comprehensive experiment regarding the robustness of both ViTs and CNNs under various existing adversarial attacks to understand the underlying reason favoring their robustness. Based on the drawn insights, we then propose a dedicated attack framework, dubbed Patch-Fool, that fools the self-attention mechanism by attacking its basic component (i.e., a single patch) with a series of attention-aware optimization techniques. Interestingly, our Patch-Fool framework shows for the first time that ViTs are not necessarily more robust than CNNs against adversarial perturbations. In particular, we find that ViTs are more vulnerable learners compared with CNNs against our Patch-Fool attack which is consistent across extensive experiments, and the observations from Sparse/Mild Patch-Fool, two variants of Patch-Fool, indicate an intriguing insight that the perturbation density and strength on each patch seem to be the key factors that influence the robustness ranking between ViTs and CNNs. It can be expected that our Patch-Fool framework will shed light on both future architecture designs and training schemes for robustifying ViTs towards their real-world deployment. Our codes are available at https://github.com/RICE-EIC/Patch-Fool.
Open AI forgot to be Thorough
Great post by Mathis Lucka about why it's important to check for "trivial things" when deploying systems. Applies to all tech, not just AI.
This isn't to call out OpenAI. It's an example of how the data science profession fell prey to the generative AI hype and forgot about being thorough. It's also a story about how a single character can change the outcome of an experiment. OpenAI published a notebook on fine-tuning GPT-3.5 for RAG. Their premise? Out-of-the-box GPT-3.5 will always try to answer a question, even if the provided context doesn't have the answer. One of the issues in their code? Their evaluation checked for "i don't know" whereas GPT-3.5 answered with "i don't know." (pay attention, the difference is subtle) 🔍
Software Content
How Instagram Grew to 14M with ONLY 3 Engineers
How Instagram Grew from 0 to 14M users in only a year. The best part? It was only done by 3 software engineers. In this video, I showcase the infrastructure and techniques used by the Instagram Software Engineers to handle one of the fastest growing applications in history.
This Is Why You Dont Outsource Your Network Security
In this video I discuss the Cisco critical vulnerability (CVE-2023-20198) that has been used to take over more than 10k network devices and many more are likely vulnerable (and will probably remain vulnerable) this vulnerability is only exploitable if the admin login portal for the web IU is bound to a public IP WHICH SHOULD NEVER BE DONE EVER UNLESS YOU ENJOY HAVING YOUR ENTIRE NETWORK HACKED!
If you liked this article and wish to share it, please refer to the following guidelines.
If you find AI Made Simple useful and would like to support my writing- please consider becoming a premium member of my cult by subscribing below. Subscribing gives you access to a lot more content and enables me to continue writing. This will cost you 400 INR (5 USD) monthly or 4000 INR (50 USD) per year and comes with a 60-day, complete refund policy. Understand the newest developments and develop your understanding of the most important ideas, all for the price of a cup of coffee.
Reach out to me
Use the links below to check out my other content, learn more about tutoring, reach out to me about projects, or just to say hi.
Small Snippets about Tech, AI and Machine Learning over here
AI Newsletter- https://artificialintelligencemadesimple.substack.com/
My grandma’s favorite Tech Newsletter- https://codinginterviewsmadesimple.substack.com/
Check out my other articles on Medium. : https://rb.gy/zn1aiu
My YouTube: https://rb.gy/88iwdd
Reach out to me on LinkedIn. Let’s connect: https://rb.gy/m5ok2y
My Instagram: https://rb.gy/gmvuy9
My Twitter: https://twitter.com/Machine01776819
Thanks for the shout!
I am really looking forward to working with you on our upcoming piece.