Building my own AI context system
Or: vibe coding my way to better conversations, again
I don’t know about you, but as AI models get better, I find myself asking for more advice and wishing for a way to give them better context on my life. It’s not the first time I’ve tried to build this, but this time I’m making more progress than usual.
What I wanted:
Running context that’s easy to share between different AI models
The “chat interface” experience - separate conversations with memory I don’t have to manually manage
Automatic summaries of recent chats/journals so I’m not mentally carrying everything or correcting wrong assumptions
Something that works on both laptop and mobile
To chat with Claude and GPT at the same time (new desire - they’re both so good and so different now)
AI coding keeps being remarkably hard and remarkably easy. Easy in that I can set up complex systems I don’t fully understand. Hard in that I still need intuition about who to ask what and when.
My current setup:
Obsidian for journaling (daily notes in Markdown)
Downloaded all my Claude (61MB) and ChatGPT (55MB) conversations - you can only get these from the webapp manually
Installed Open WebUI on a Hetzner server with API keys for OpenAI and Anthropic
(When I say “I installed,” I mean Claude Code SSH’d in and did it)
Uploaded 6 months of journals and chats
Set up Tailscale so the server is only accessible via my private network (not exposed to the internet)
Where I got stuck:
Choosing the shape of the setup - there are a million possibilities, I don’t know enough about each one. Did a mix of trusting Claude/GPT and just trying things. Got stuck with another chat interface, switched to Open WebUI.
Copy-pasting errors and commands between webapp UI and terminal
Trying to set up VSCode to SSH into the server
Wrong model name for the Anthropic API (Claude Code didn’t know about the reasoning model yet)
Too many files for journals/chats - had to get Claude to clean and merge them
Security/privacy concerns I’m still not totally confident about
Trying to setup Cloudfare so that only my email can log in and failing for like an hour
The takeaway: This took a few hours of vibe coding. There are a thousand ways to do this. If you know more about cybersec than I do and have suggestions for improvements, please let me know.