Aditya Arya
All writing

Two AI Agents, One Linux Account: A Postmortem

On the last day of the ML Challenge, someone else's AI agent logged into my account on a shared college GPU server, read my code, deleted my work and kept killing my training jobs. How it got in (my fault, mostly), how we caught it, and what it cost.

· 11 min read

At 07:57 on the final day of the ML Challenge, a second AI coding agent started working inside my project.

It wasn't mine.

By the time I understood what was going on, it had read my code and my backups, run its own analysis on my data, replaced my source files, deleted five directories and was sending kill signals to my GPU jobs every time they restarted. Meanwhile my own agent, in the same account, was trying to finish the competition.

Two AI agents, one Linux user, one deadline. 2026 is a weird year.

This is the postmortem. It's written the way I wish more incident reports were written: what happened, how we found it, how it got in (spoiler: through my own carelessness), what it cost, and what I'd tell anyone who works on a shared server. I've reported the full details, with evidence, to the server admins. This public version leaves out IP addresses, hostnames and account names, and it doesn't accuse anyone. More on that below.

TL;DR

  • What: someone logged into the server through another student's account, used su to switch into my account with my password, and started an AI coding agent in my competition project.
  • How: my password still followed the predictable default pattern every assigned account was given. I knew the pattern and never changed it, because "why would anybody do this?" Somebody did.
  • Damage: code replaced, work directories and a backup deleted, training jobs killed repeatedly, my solution read hours before the deadline.
  • Saved by: my own agent evacuating the git history and caches into hidden directories 30 minutes into the attack, plus a backup on my laptop.
  • Cost: about six hours of the final day. We finished 101st on the public leaderboard, one place outside the top 100.

Background

Team Syndicate Smashers was in the final day of the ML Challenge 2026, an entity resolution problem over 1.7M businesses (the full story is in the other post). We were training on our college's shared GPU server: one machine, many students, one account each. Accounts were handed out with passwords that all followed the same pattern.

My setup that morning: a Claude Code session running from my laptop, doing the heavy lifting in the project directory, with long GPU jobs detached in the background. Our plan for the day was a self-training round for France, about two hours of compute, then the final submissions.

What Happened

All times are IST.

DAY BEFORE (26 Sep)
17:22   someone logs into MY account directly, from a machine that isn't mine or my teammate's
17:31   same machine logs in again; a VS Code client connects and installs its server
        on my account for the first time (a VS Code version that had never connected before)
17:37   they leave. Nothing visibly broken. Nobody notices. (Foreshadowing.)

FINAL DAY (27 Sep)
07:55   an SSH login as ANOTHER student's account, from that same machine
07:57   inside that terminal:  su <my account>  → types my password → starts an AI agent
        in my project directory
08:05   the agent checks its Python packages, prints my requirements.txt,
        lists my backup folder, reads one of my pipeline scripts
08:24   the agent runs statistics over my training and test data
08:27   MY agent, noticing files changing under it, clones the repo (full git history) into a
        separate hidden directory; 08:30, it starts a separate cache directory too
 ...    my predict.py is replaced; work/, output/, dataset/, code/ and my backup folder
        are deleted; my running jobs start dying with SIGTERM, again and again
~10:00  we find the foreign process, freeze it, kill its children, kill it
~10:45  password changed, recovery plan written
15:40   pipeline rebuilt from surviving caches, back where it should have been at ~10:00
23:16   final submission. Rank 101.

How We Found It

The first symptom was that the ground kept moving under my own agent. Files it had just written were different. work/ was gone. Jobs that had been running for an hour died with SIGTERM, restarted, and died again. My agent reported that predict.py had been replaced with a different program, which is not the kind of thing you want to read on deadline day.

The investigation itself was boring Unix, which is the best kind:

ps -u <me> -o pid,ppid,sid,tty,etime,args   # two `claude` processes, not one
pstree -p <pid>                              # where did the second one come from?

The second agent's process tree told the whole story:

sshd
 └─ sshd
     └─ sshd
         └─ bash                ← logged in as ANOTHER account
             └─ su <my account> ← typed my password
                 └─ bash
                     └─ claude  ← their AI agent, running as me

Mine was on one terminal from my laptop. Theirs was on a different terminal from a different machine, wrapped in a su. Each Claude Code process also carries its own session ID in its environment (/proc/<pid>/environ), which made it trivial to tell the two agents apart.

Containment

The rule was simple: kill theirs without touching mine. My session had hours of state in it.

kill -STOP <their claude>        # freeze it first, so it can't react or respawn anything
pstree -p <their claude>         # list what it was running
kill -TERM <child shells...>     # stop its children (one was tailing my normalize log)
kill -KILL <their claude>        # then the agent itself
ps -p <their claude>,<my claude> # verify: theirs gone, mine alive

Then I changed the password and checked ~/.ssh/authorized_keys (empty) and crontab -l (empty).

The Detective Part (a.k.a. the environment leaks)

This is the bit I found genuinely mind-boggling.

su <user> without the dash switches your identity but keeps most of your environment variables. So when the intruder became me, their agent carried their environment into my account.

The foreign agent's very first environment check, at 08:05, contains this line:

ls: cannot access '/scratch/<another-student>/.../cache/huggingface': Permission denied

An agent running as me went looking for a Hugging Face cache inside another student's private scratch directory, because that's where its inherited environment said the cache was. It failed, because as me it had no permission to go there. That one line of an error message ties the agent back to the account it was launched from.

Other traces the login records kept:

  • The login that opened the intruder's terminal happened two minutes before the agent's session files were created.
  • The day before, a VS Code client connected to my account and installed a server build that had never connected before, in the exact minute of that unknown login.
  • The agent's working files sat right there in /tmp, with timestamps. Its conversation transcript, interestingly, did not.

Attribution (or: why I'm not naming anyone)

The records show which account the intruder's terminal was logged in as. They don't show who was at the keyboard:

  1. If my default password was guessable, so was everyone else's. The account used as the launch pad may itself have been borrowed.
  2. The machine it came from was used by several accounts, so it looks like a shared lab machine or gateway, not one person's laptop.

So I did the boring, correct thing: wrote a full report with the evidence and asked the admins to confirm it from the logs only they can read (auth.log records exactly who ran su to my account, and when). Internet mobs are not an incident response process.

Root Cause: Me, Mostly

Let's be honest. The accounts were handed out with a predictable password pattern. I knew the pattern. I knew anyone else on the server knew it. And I didn't change mine, because who on earth would log into a random student's account on a college server?

Someone did, a day before the deadline, and they didn't even have to guess hard.

The server made it easy too: password SSH logins were on, su between student accounts was unrestricted, and nothing alerted anyone to a new login. But the first and cheapest line of defense was mine, and I skipped it. It took one passwd command, about ten seconds.

What It Cost

  • About six hours of the final day. Our round-2 run was supposed to finish around 10:00. It finished at 15:40, after rebuilding caches, re-running 83M stage-1 pairs and 8.6M cross-encoder pairs on shared GPUs.
  • Deleted work: caches, outputs, a dataset copy and a backup folder. Nothing permanent, thanks to git and the laptop.
  • Our solution, read by someone else's agent hours before a ranked deadline.
  • Rank 101. The top-100 cut was one place away.

Here's the painful arithmetic. All of our big France gains came in the last ~6 hours of the day, and the final steps were each worth about +0.0001 to +0.00025 for 1–2 hours of compute. The gap between 101st and 100th was almost certainly smaller than one of those steps. I can't prove what those six hours would have bought. I also can't stop thinking about it.

What Went Well

  • My agent evacuated before the deletions reached it. At 08:27, half an hour into the intrusion, it cloned the repository with its full history into a separate hidden directory, and at 08:30 it started a separate cache directory. The intruder's agent kept working in the original project folder and never touched them.
  • A backup on my laptop with everything up to our best submission at the time.
  • Caches in their own directories, so we rebuilt from the intermediate results instead of from scratch.
  • Freeze first, then kill. SIGSTOP before SIGKILL meant the foreign agent couldn't react while I read its process tree, and my own session survived untouched.

What Went Wrong

  • A default password, on a shared machine, with a deadline. Enough said.
  • Nobody noticed the day before. Two logins from an unknown machine, a new VS Code build installed, and I didn't look at last once.
  • I didn't watch for other sessions on my own account until files started disappearing.

Where We Got Lucky

  • The intruder's agent didn't find or wipe the hidden directories.
  • The laptop backup was recent enough to contain our best submission.
  • It happened in the morning, not at 22:30.

The AI-Agent Angle

This part should worry more people than it does.

An AI coding agent with shell access is you, as far as the server is concerned. It has your permissions and your files, and it runs rm -rf without a second thought if that's what its task implies. My agent could have done the same damage to my project; it just had better instructions.

The intruder didn't need to be a Linux wizard. They needed one password and one sentence to an agent. The agent did the rest at machine speed: read the project, understood the pipeline, ran the data, "cleaned up" directories, killed "stale" processes. From the agent's point of view it was probably just being helpful in a project it believed it had been asked to work on.

In a world where everyone runs agents on shared machines, the blast radius of a stolen password is the blast radius of an agent that has it. Account boundaries are now the only boundaries that matter, so treat them like it.

If You Use a Shared Server, Do This Today

passwd                          # yes, now. Not after the deadline.
last -F -w $USER | head -20     # do you recognize every address?
who; ps -u $USER -o pid,tty,etime,args   # anything running that isn't yours?
ls -la ~/.ssh/authorized_keys; crontab -l   # anything you didn't put there?
  • Use SSH keys, and ask your admins to turn off password logins.
  • Keep your git history outside the working directory, and push it somewhere private.
  • Keep a backup off the machine after every milestone.
  • If you're an admin: force a password change at first login, restrict su (pam_wheel), set PermitRootLogin no, and turn on auditd.

The End

So yeah. We lost six hours, a backup folder and maybe a top-100 spot, and gained a very expensive lesson that cost ten seconds to avoid. The worst part isn't that someone got in. It's that I knew exactly how they could, and decided it wouldn't happen.

It happened.

Thanks for reading. If you're a student on a shared server, go run passwd. I'll wait.

And if you're a recruiter who read this far: I resolve entities, I write postmortems, and I now change my passwords. HIRE ME PLEASE🥺.

Checkout my GitHub, LinkedIn, X. bye.