From "Mystery Menus" to Software 3.0: The End of the Middleman

Ever been stuck at a restaurant staring at a menu full of names like Xiao Long Bao or Mapo Tofu, unsure of what to expect?

We’ve all felt that “mystery menu” hesitation, but Gemini is changing the game by turning text into a visual experience.

By simply using your camera, Gemini identifies traditional dish names and instantly generates high-quality images alongside estimated calorie counts, effectively bridging the gap between cultural heritage and modern convenience.

The jump from a text-heavy, traditional Asian menu to an AI-enhanced visual guide isn't just a "cool feature", it’s a living demonstration of the next major era in computing.

In a recent talk at Sequoia Capital, Andrej Karpathy (former Director of AI at Tesla and co-founder of OpenAI) explored the transition from Software 1.0 to Software 3.0, and he used a "Menu Generator" app as the perfect example of why the software we build today is about to become obsolete.

[05:40] The Three Eras of Software

To understand why the "Gemini-enhanced menu" is a glimpse into the future, we have to look at how software has evolved:

• Software 1.0 (The Era of Code): Humans write explicit, line-by-line rules. If you wanted to build an app that adds pictures to a menu, you’d have to write code for OCR (text recognition), code to call an image database, and code to render the UI.

• Software 2.0 (The Era of Data): We stopped writing the rules and started training neural networks with data. Here, the "programming" was about curating massive datasets to teach a model how to recognize a dish like Mapo Tofu. [02:55]

• Software 3.0 (The Era of the LLM Interpreter): In this new paradigm, the LLM (like Gemini) acts as the computer itself. You don't build an app, you provide a "prompt" or "context" (like a photo of a menu) and the model performs the computation directly in the digital information space. [03:22]

The "MenuGen" Epiphany

Karpathy shared a story about building an app called MenuGen.

He "vibe coded" a complex system that took a photo, extracted the text, generated images, and rendered a new UI on Vercel. But he soon realized this app "shouldn't exist." [06:04]

The Software 3.0 version is much simpler:

You give the photo to a multimodal model and say, "Overlay pictures of these foods and their calorie counts back onto this menu."

In this model, the "neural network is doing more and more of the work," and the need for a "middleman" app or complex UI code disappears.

The model takes the raw pixels of your menu and returns the enhanced pixels directly. [06:11]

Why This Matters for the Future

The transformation of a restaurant menu is a micro-example of a macro-shift:

1. Agentic Engineering

We are moving from "Vibe Coding" (getting a model to write code for us) to "Agentic Engineering."

Instead of writing software to solve a problem, we are building "agents" that use their own intelligence to navigate environments, debug on the fly, and deliver results. [16:28]

2. Jagged Intelligence

AI models today are "jagged", they can refactor 100,000 lines of code but might struggle with simple logic (like how many letters are in "strawberry").

Because of this, the human role shifts from "coder" to "director." [11:42]

3. Outsourcing Thinking vs. Understanding

As Karpathy noted, "You can outsource your thinking, but you can't outsource your understanding." [28:10]

In the world of Software 3.0, your value isn't in knowing how to write a Python script to fetch calorie data, it's in the taste, judgment, and design required to direct an AI to create a useful experience for a diner. [21:52]

Final Thought

The next time you see Gemini instantly transform a confusing menu into a visual guide, remember:

you aren't just looking at a better app.

You're looking at the death of the "app" as we know it, replaced by a raw, intelligent interface that understands the world as well as we do.

#Software30 #AndrejKarpathy #AI #GeminiAI #FutureOfTech #AgenticEngineering