J.A.R.V.I.S.

What it is

A hybrid local and cloud AI system for home automation and personal productivity, built from scratch in Python. It started on a laptop with 4 GB of RAM and an integrated CPU; today it runs on a Ryzen 5 3600 with an RTX 2060.

How it thinks

Every request first goes through deterministic routing: home automation commands, apps, reminders and documents each have their own path. Only what's left reaches a language model, the local one (Qwen2.5 7B) or, for complex questions, the cloud.

Principles

Local by default, cloud only when needed. Personal data never leaves the PC. Any external service can go missing without bringing the rest down.

Six paths for every request

Before bothering a language model, J.A.R.V.I.S. tries six paths in a fixed order: home automation, opening apps and audio control, reminders, personal documents, local model, cloud. If the cloud doesn’t answer it falls back to the local model on its own, with a sarcastic remark instead of an error.

A memory that actually remembers

In the background it summarizes conversations, always locally, so it can answer “what did you suggest for my thesis?”. If the PC stays off for days, it catches up on the whole backlog when it restarts. Time labels like “yesterday” or “3 days ago” are computed by the code, not the model: before that, the model described a three-minute-old memory as “yesterday”.

Proactive, not just reactive

It delivers reminders without being asked and cross-checks deadlines with the weather forecast (“you have a deadline tomorrow and it’s going to rain, shall I remind you earlier?”). The weather decision is deterministic, no AI involved. It also warns when the disk fills up, when the local model stops responding and when the cloud’s free quota reaches 80%.

Privacy by architecture

Documents and memory are processed only locally, never in the cloud. Guest users can’t see the Admin account’s data. Requests for passwords to other sites always get a clear refusal.

A voice and a face

The voice goes through a chain of audio effects (filter, compressor, reverb, one semitone lower, a light chorus) that brings it closer to the film’s. It can be interrupted with a keyboard shortcut: the first version used the wake word, but the microphone picked up J.A.R.V.I.S.’s own voice and it kept interrupting itself. The HUD, with “arc reactor” rings and an archive of 3D holograms, reacts to the voice in real time; the 3D models come from a hand-written STL writer or are downloaded from Thingiverse.

Measured choices, not hunches

Qwen2.5 3B was faster, but lost the formal register and fluent Italian: rejected. Qwen3 4B wrote its own reasoning in English inside the answer; Qwen3 8B was 13% slower. I also evaluated streaming responses and AI-generated 3D, and rejected both with the numbers in hand.

The bug that shut down my PC

While building the “shut down the PC” command, a test really shut down the machine, even though the functions were supposedly mocked. The cause was a table of functions built at startup that the tests couldn’t replace. Restart and shutdown were removed; the lesson was learned.