The Bhagavad Gita
of Data Engineering

Five thousand years ago, on the field of Kurukshetra, the greatest archer of his generation put down his bow.

Arjuna was not undertrained. He was not outnumbered in any way that mattered. He simply looked at what was in front of him and could not find a version of himself that survived the next ten minutes intact. His hands shook. His mouth went dry. The Gandiva — the bow that had never left his grip in battle — slipped.

Sound familiar?

In 2026, the data engineer arrives at a similar moment, usually somewhere between the second coffee and the first stand-up. You've spent years — maybe decades — on SQL, Python, Spark, Airflow, dbt, Terraform. You've moved petabytes. You've debugged a join at 3 a.m. while your partner asked, reasonably, what a "skew" was. You earned your place.

Then a headline informs you that your entire discipline has been deprecated. Your VP forwards it. With a smiley.

Arjuna, to be fair, had it easier. His enemies at least stood still long enough to be counted. Yours ship a new minor version while you are reading about them.

And the anxiety is not irrational. The tools are being abstracted. The grunt work that quietly taught a generation its judgment — the tedious profiling, the manual reconciliation, the afternoon lost to a timezone bug — is being automated away before the next generation can be bored by it. That is a real loss and pretending otherwise is its own kind of denial.

So the question is not whether the battlefield changed. It changed. The question is the one Krishna answered: what do you do while standing on it?

Notice what Krishna does not do. He does not tell Arjuna to leave the field. He does not tell him to try harder. He does not send him a link to a course.

He gives him something rarer: a way of acting clearly when the outcome is genuinely uncertain. Seven hundred verses, eighteen chapters, and not one of them is a tooling recommendation.

Which is convenient, because your tooling recommendation expires on Thursday.

Arjuna's crisis was never tactical. It was about identity. He was a warrior. Everything he believed about himself was welded to being the best at one specific thing. The moment that thing stopped guaranteeing a good outcome, he collapsed — not physically, existentially.

This is what FOBO — the Fear of Becoming Obsolete — actually feels like. It is not the fear of being fired tomorrow. It is the low hum of suspecting that your skills are depreciating faster than you can amortise them, and that the definition of "relevant" is being rewritten by people who have never had to explain a Kafka consumer lag to a CFO.

The feed does not help. The feed is designed not to help.

By the time that number reaches your feed it will have become "52% of engineers have already been replaced," attributed to a report nobody has opened, in a carousel with eleven slides and one spelling mistake.

Here is what Krishna understood and the feed does not: the paralysis is the problem, not the battlefield. Arjuna was not defeated by the opposing army. He was defeated by a fixed idea of who he was. The moment he defined himself as "the archer" rather than "the one who acts rightly," he was stuck — and no amount of additional archery was going to unstick him.

The engineer who defines themselves as "the Spark person" is making precisely the same move, with a smaller bow.

Every data team I have worked on has contained the same three people. Names change. Behaviour does not. They will keep turning up in this article, because they keep turning up in your standup.

Nobody remembers what Tambi solved in 2014. Nobody has finished anything Rao started. And nobody, in the history of the enterprise, has answered Subramaniam's second question — "Who consumes this data?" — on the first attempt.

Frankl, writing out of Auschwitz, reached the same place Krishna took Arjuna on the chariot: you do not control the field, you control your response to it. The Stoics called it the dichotomy of control. The Gita calls the person who has mastered it Sthitaprajña — of steady wisdom.

Your feed calls it "building in public." It is not the same thing.

The bow is not the archer. This is not a metaphor I invented; it is the oldest one in the discipline. Arjuna's Gandiva was a magnificent weapon. It was also, in the end, a stick with a string, and the entire Gita happens because a stick with a string cannot tell you what is worth doing.

Data engineering has run this experiment about once every six years, in public, with everyone's résumés attached.

Every one of those technologies was, at launch, the end of history. Every one of them was going to make the previous thing a curiosity. Every one of them had a conference, an evangelist, and a certification track priced like a small holiday.

And every single time, the engineers who came through the transition intact were not the ones who had memorised the tool.

Across all of it, the durable skill list has barely moved:

How data actually flows. How systems actually fail. What "quality" means when two departments define a customer differently and both are right. Latency and what people will pay for it. Semantics. Trade-offs. Where the business keeps its real numbers, as opposed to the ones in the dashboard.

None of those appear on a certificate. All of them survive the next migration.

The Gita describes three qualities — Gunas — woven through all of nature and therefore through all human behaviour: Tamas (inertia, dullness), Rajas (restless activity, craving), and Sattva (clarity, harmony). They are not personality types. They are weather. Everyone passes through all three, usually before lunch.

But in a disruption, each one produces a very specific kind of engineer. You have met all three. You have been all three.

Read that last word again: bind. All three bind. Sattva is the pleasantest rope, but it is still a rope. Which is the part the LinkedIn version always leaves out, because "achieve clarity, then transcend even that" does not fit in a carousel.

The practical point for a career is smaller and more useful. Most of us do not sit in one Guna. We oscillate. Tamas on Monday, when we ignore the migration email. Rajas on Tuesday, when we enrol in three courses at midnight. Tamas again on Wednesday, when we do not open any of them. And on Thursday we read layoff threads until we feel briefly, horribly awake.

Both moods feel like reactions to the outside world. Both are actually the same trap: paralysis by denial, and paralysis by noise. Neither of them writes a line of production code.

Sattva is not enthusiasm and it is not detachment-flavoured indifference. It is discernment — the ability to look at a hundred announcements and ask which two of them change how you would design a system, then let the other ninety-eight go past without lodging in your nervous system.

If the Gita feels far away, notice how many traditions walked to the same conclusion without consulting each other. The insight is not Indian or Western. It is what humans work out whenever the ground moves.

This is the most quoted verse in Indian philosophy and probably the most misread. People take it as "do not care about results," which would be a licence for mediocrity, and Krishna is not in the business of licensing mediocrity. Note the second half: nor let there be attachment to inaction. He closes the escape hatch in the same breath.

What it actually asks for is harder than either extreme: work at full intensity, full skill, full attention — and stop staking your identity on an outcome that was never in your jurisdiction. The arrow is yours. Its flight is weather, wind, and eleven other people.

Translated into a job that has stand-ups:

You may design the finest pipeline known to mankind. Idempotent. Backfillable. Documented. Tested. Alerting that pages the right person with the right context.

Then Finance buys a company.

Congratulations. Your source system is now three source systems, one of which exports to SFTP on a schedule that a man named Dieter controls personally.

Nishkama Karma is not consolation for that. It is the only posture from which you can keep building well anyway — because if your self-worth is stapled to the survival of a specific pipeline, you will either stop caring or stop sleeping, and the industry has an ample supply of both.

Be honest about what the machines now do well, because pretending otherwise is Tamas in a nice shirt.

They write SQL. They write Python. They scaffold DAGs. They generate tests — including, occasionally, tests that pass because they assert nothing, which is a very human thing to have learned. They produce Terraform. They document pipelines better than we do, mostly because we don't. They suggest schemas. They flag anomalies at three in the morning without becoming resentful about it.

That is Karma. Action. Execution. Enormous quantities of it, cheaply, at two o'clock on a Sunday.

Now the other list — the questions no model can settle on your behalf, because they are not questions about code:

Why are we collecting this at all? Is this metric measuring the thing or the proxy for the thing? Are we allowed to hold this data in this jurisdiction? Should we, even where we may? Can this source be trusted, given that the team maintaining it was disbanded in March? What breaks downstream when the model is confidently wrong? What trade-off are we accepting, and who eats the cost of it? Should this system exist?

That is Dharma. Not "purpose" in the motivational-poster sense — the older, harder sense: the right action for this person, in this role, in this situation, given the consequences they will have to live with.

Notice which participant added the most value and which one was fastest. They were not the same participant.

That is the whole shift, and it is worth stating plainly because it is easy to hear as reassurance when it is actually a warning: generating a solution is becoming cheap. Knowing which problem deserves a solution is becoming the scarce thing.

Which is uncomfortable, because scarce things are valued — and most of us were promoted for the other one.

The comfortable version of this argument is "AI can build but it cannot architect." I used to make it. I no longer think it is true, and repeating a false comfort is not wisdom, it is a sedative.

AI already participates in architecture. It proposes decompositions, spots coupling you missed, argues both sides of a build-versus-buy, and produces a decision record more legible than the one you wrote at 6 p.m. on a Friday. It will keep getting better at that.

The claim that survives is narrower and much more stubborn. Architecture in a real organisation does not rest on technical context. It rests on context that is incomplete, political, historical, regulatory, economic and ethical — and most of it was never written down, because writing it down would have required someone to admit it out loud.

The valuable senior engineer is not the one who knows the most patterns. It is the one who can tell you why the ugly decision exists — and can therefore tell you which ugly decisions are load-bearing and which are simply old.

That is not nostalgia. It is a genuinely hard inference problem over evidence that mostly does not exist in any repository. A model can read your codebase. It cannot read the 2017 conversation in which a VP promised a customer something, or notice that the reason two teams duplicate a table is that they stopped speaking in 2021.

So the honest formulation is not "AI cannot architect." It is this: architecture is a judgment about consequences inside a specific human organisation, and consequences are the one thing that cannot be generated in advance.

Your moat is not that you can draw the boxes. It is that you have watched boxes like these fail, in this company, with these people, and you remember what it cost.

Philosophy without evidence is just a nice mood. Here are the numbers I can actually source — and, because this is an article about discernment, what each one does not say.

Arjuna's despair came from seeing only the destruction in front of him. Krishna did not tell him the destruction was imaginary. He widened the frame.

Ninety-two million displaced is a real and painful number, and if you are inside it, the net figure is no comfort at all. But an engineer reading only the first half is reading the battlefield the way Arjuna read it in Chapter 1 — accurately, and incompletely.

Now read the method, because this is exactly the number that gets flattened into a carousel. It compares advertised salaries within the same occupation, depending on whether the posting mentions AI skills. It does not follow the same people before and after learning something. PwC itself is careful to say the relationship may not be causal.

Translation: the market is paying more for roles that require AI fluency. It is not a promise that adding "LangChain" to your headline produces a 62% raise, however confidently the carousel asserts otherwise. If it were, Rajas Rao would be independently wealthy.

In Chapter 2, Arjuna asks a very practical question. Not "what is truth" — he asks what such a person is actually like. How do they sit? How do they speak? How do they move through the world? Krishna's answer is the description of the Sthitaprajña, and it is one of the most precise passages in philosophy.

Untroubled in sorrow. Sees the layoff thread. Feels it — Krishna never asks for numbness — and does not restructure their life at 2 a.m. on the strength of a headline written to be restructured around.

Free from longing in pleasure. Does not need the dopamine of being first to the new framework. Can watch a launch week go past without acquiring a personality.

Without attachment. Not defined by employer, stack, or title. The identity is portable because it was never rented from a vendor.

Without fear. Treats AI as a very fast, very literal colleague with no memory of last quarter — useful, not adversarial, and never left unsupervised near production.

Without anger. Does not rage at the industry, at model vendors, or at juniors "who just prompt." That energy has exactly one productive destination, which is building something.

To be clear, because the internet flattens this too: the Sthitaprajña engineer still panics occasionally. Something goes wrong at 4 p.m. on a Friday and something in the chest goes cold. That is a nervous system, not a character defect.

The difference is what happens next.

Not scripture. Field notes.

I want to be careful here, because this joke has a cruel version and a true one.

The cruel version sneers at anyone learning in public. That is worth nothing. Half the good engineers I know started by posting something slightly embarrassing.

The true version is narrower: a certificate proves you finished a course. Production proves whether you understood it. Both are real signals. They are just not the same signal, and only one of them keeps its value after the vendor rebrands the product.

Verse 2.47, applied to your feed: the post is the fruit. The build is the action. Krishna's advice on which of the two to attach yourself to has been sitting there, unread, for five thousand years, currently at zero reactions.

Every few years the industry arranges itself into two armies and insists you pick a side. The banners change. The formation does not.

Both armies are wrong in the same way, which is the part nobody puts on a banner.

The old world says the new tools are toys. The new world says the old work was never necessary. Neither camp has to maintain the result on the 4th working day of the month while Germany waits.

Arjuna's question — which framework should I learn? — is not a stupid question. It is a perfectly sensible tactical question asked in place of the strategic one, which is what makes it dangerous. Krishna does not answer it, and does not answer it deliberately. Chapter 2 begins with him declining to accept the terms of the question at all.

The better question, the one that survives every banner change: what am I responsible for, and what does doing that well actually require of me now?

All philosophy is theory until the phone lights up.

Every generation of tooling has promised to end that conversation. None of them has. The stack traces get prettier. The 3 a.m. remains.

And here is what actually separates the engineer who is useful at 3 a.m. from the one who is merely awake: it is not knowledge of the system. Both know the system. It is that one of them is diagnosing and the other is reacting — restarting things, escalating, apologising in a channel, rerunning the DAG in the hope that the universe reconsiders.

Panic is not a moral failure. It is just a bad debugging strategy. It narrows attention at exactly the moment the answer is somewhere peripheral — a merged PR, a rotated credential, a "minor schema tweak, no impact."

This is the entirely unmystical, deeply practical core of Sthitaprajña. Krishna is not selling serenity as a lifestyle. He is pointing out that a mind churning about outcomes cannot see what is in front of it.

Here is the part people forget about the Gita, usually because they stopped at the quotable verses.

After seven hundred verses, Arjuna does not renounce the world. He does not retire to a cave. He does not pivot into advisory work.

He picks up the bow.

The same bow. The same field. The same eleven armies that terrified him in Chapter 1 are still standing there in Chapter 18, entirely undiminished by the philosophy. Nothing external changed at all.

Sthito'smi gata-sandehaḥ — I stand firm, my doubt gone. Not "I am certain of the outcome." Doubt about the outcome was never the problem. What left him was doubt about what he was there to do.

So: open the IDE. Open the architecture diagram. Open the AI assistant as well — refusing to is not integrity, it is Tamas with better branding.

Use everything available. Let it draft the SQL. Let it write the DAG, the Terraform, the tests, the docstring you were never going to write. Let it read the log you cannot face at 3 a.m. One day it will draft half the architecture diagram, and that day will arrive earlier than the conference talks predict.

Just be clear about who is supposed to be deciding.

Because wisdom is not generated from syntax. It comes from having watched systems fail in ways the design review did not anticipate. From organisations that reorganised around the thing you built. From metrics that were technically correct and quietly lying. From customers who behaved in a manner no schema permitted. From consequences that arrived three years after everyone in the design meeting had changed jobs.

That is experience. Compressed, unglamorous, largely undocumented, and yours.

That is judgment.

That is your Dharma.

Pick up your bow.

Production is waiting.

And somebody, naturally, has already changed the schema.