In my previous post about agentic vim, I wrote about how to integrate AI agents into my nvim environment using the Agent Client Protocol (ACP) and codecompanion. In that setup, nvim acts as the client, prompting the agent and bringing the responses back into the editor buffers.

But what if we turn things around?

What if the AI agent is the driver, and we want to let it interact with our running editor session? E.g., if I’m running an external agent CLI or a pair-programming assistant, and I want it to be able to open a file at a specific line, check LSP diagnostics to see if its changes compile, populate my quickfix list, or send me notifications and let my entire vim flash by toggling color schemes?

I didn’t find a solution, so I built nvim-editor-mcp.

The Problem

• External AI agents or CLI assistants operate blindly, isolated from my running editor session.
• When working on shared coding tasks, the agent cannot see what I see. It doesn’t know my active buffer, cursor position, or the LSP diagnostics currently flashing on my screen.
• Instead of a shared, interactive coding session, the workflow becomes disjointed: the agent outputs file paths and line numbers in the terminal, and I have to manually navigate to them.
• I want a seamless, bi-directional partnership where the agent can interact directly with the Vim API to inspect my context in real time and guide me through the code.
• And, of course, I want this integration to be loose and lightweight, without forcing the entire agent runner to live inside a heavy nvim plugin.

Solution

An MCP server that exposes nvim operations as tools, bridging the agent’s requests to nvim via RPC.

I created nvim-editor-mcp. It is a lightweight Python server.

Building it was pretty straightforward, mainly because of two very useful abstractions.

• fastmcp abstracts away the entire MCP boilerplate. It handles JSON-RPC serialization, protocol lifecycles, and tool schema generation. You define your tools with a simple @mcp.tool decorator, and fastmcp takes care of the rest.
• pynvim provides the Python interface to nvim’s Msgpack-RPC API. Instead of worrying about socket-level message passing, it gives us a clean, high-level API to command the editor, run Lua code, and query buffer/diagnostic states directly.

Together, they make bridging the agent protocol and editor state a breeze.

How it connects

nvim communicates over a Unix socket. When you open a terminal buffer inside nvim, it automatically sets the $NVIM environment variable containing the path to that socket.

nvim-editor-mcp is designed to be zero-config:

1. If started from inside nvim (e.g., via a terminal buffer or an integrated runner), it automatically reads $NVIM.
2. If started externally, it falls back to connecting to a specified socket, which can be identified by the agent

The Tools

Once connected, the server exposes the following tools to the AI agent:

Because we expose nvim_exec_lua, the agent has a powerful fallback to do basically anything it wants inside our editor session. For example, to check the path of the current buffer, the agent can call nvim_exec_lua with:

return vim.api.nvim_buf_get_name(0)

Client Configuration

To hook it up to an MCP client like Claude Desktop or any other MCP-enabled AI CLI runner, just add it to your configuration file:

{
  "mcpServers": {
    "nvim-editor-mcp": {
      "command": "nvim-editor-mcp"
    }
  }
}

(Ensure nvim-editor-mcp is on your $PATH, which you can do easily by installing it with uv tool install git+https://github.com/gierdo/nvim-editor-mcp.git).

tl;dr

• Exposing local development environments to AI agents via MCP enables real-time, interactive pair-programming sessions between you and your agent.
• nvim-editor-mcp lets external agents drive your editor and see what you see by querying the nvim API directly (buffers, cursors, diagnostics).
• fastmcp and pynvim act as clean abstractions, making it simple to bridge the Model Context Protocol to nvim’s RPC interface.
• Zero-config connection discovery over local Unix sockets handles the plumbing automatically.

Since about the first year of my studies I’ve been using git. I manage and version my personal documents, letters, system configuration, notes, tax declaration (oioi, I should really finish last year’s taxes soon..), basically my digital life with git, using a little server with gitolite as home-local server and synchronization point where needed.

Additionally, of course, I’ve been working with git. Basically every project I worked with in the last many years was versioned with git, with the exeption of perforce being used by my first employer (🤮, not the employer, great people. I had a lot of fun there and learned a lot. But perforce…)

Anyway, what I am going for is, I have used git for a long time. I have understood git for a bit less time, I have been confused by git many times and I spent a lot of time trying to resolve merge conflicts. I think I have a moderately solid understanding of the basic concepts of git, I know implementation details and I can explain the difference between a git merge and a git rebase. For the last few years, I have just simply used git without thinking about it and adapting my basic workflow.

Until about 3 weeks ago.

I have heard of worktree before, but never passed the stage of “Huh, interesting, I have to look into that one day.”

Whenever I wanted to quickly fix something on main, wanted to review a specific branch or for whatever other reason do something outside of the context of the branch I am currently working on, I would stash / commit my current changes, switch branches, do the thing, switch back and either pop the stash or reset my branch / amend to the temporary commit.

Whenever I needed parallel states of a repository, e.g. because binary files have to be examined or for a bit of crude copy paste action or parallel work or whatever, I would create physical copies of the repository in the same top level directory, à my_repo and my_repo_copy. Which is a bit sh*t, for several reasons. Nobody needs several identical index directories, and it’s just confusing as well as a lot of manual action.

The Problem

I have to or want to quickly work on a different branch than my current working state, or I want to work with several branches at the same time.

I don’t want to create intermediate commits, stash and pop, manually copy paste and create error prone workflows that need manual cleanup.

Solution

git worktree!

Manage multiple working trees attached to the same repository.

A git repository can support multiple working trees, allowing you to check out more than one branch at a time. With git worktree add a new working tree is associated with the repository, along with additional metadata that differentiates that working tree from others in the same repository. The working tree, along with this metadata, is called a “worktree”.

What does that mean?

I have a repository, e.g. the one with this content from github.

It’s cloned locally in gierdo.github.io and checked out to master. Now, I want to work on the new article about worktrees in the branch worktrees. I could run a git checkout -b worktrees or git checkout worktrees and switch branches on my “main worktree”, which is, well, the main worktree you always get when you clone or init a repository.

Or I could checkout the branch to a “linked worktree” under a specified directory, reusing my entire rest of the setup and the already present .git/, with git worktree add -b worktrees place_for_my_worktrees/worktrees or git worktree add worktrees place_for_my_worktrees/worktrees

This creates a directory place_for_my_worktrees/worktrees, which contains the (you guessed it) linked worktree.

Pretty handy!

To make it even more handy for me and support shell completion including fuzzy searching, I configured a bit of sugar around it.

In my global gitconfig, there are two alias configurations, allowing me to make sure that all worktrees are added under the same repository-local directory (worktrees/):

[alias]
  worktree-add = "!f() { git worktree add -b \"$1\" \"worktrees/$1\"; }; f"
  worktree-checkout = "!f() { git worktree add \"worktrees/$1\" \"$1\"; }; f"

My global gitignore contains the woktrees directory, as my worktrees are in-tree of the parent “main tree”. Yes, this is potentially a bit dangerous, but also very convenient. I’m a bit undecided about this, but in all 3 weeks of experience with it I haven’t encountered any issues, and it’s easy enough to change if needs be..

...
tags
.tool-versions
.vim
worktrees
...

My custom zsh completion configuration contains the completion definition to complete branches for the git worktree checkout, which makes it behave like a normal git checkout, including fuzzy completion if configured correctly.

#compdef git-worktree-checkout

_git-worktree-checkout() {
    _values 'branches' ${${(f)"$(git for-each-ref --format='%(refname:short)' refs/heads refs/remotes 2>/dev/null)"}:-""}
}

If you are interested in the full configuration, it’s all here.

tl;dr

• sometimes you need several branches or you have to quickly switch branches
• workarounds are possible
• workarounds are workarounds
• git worktree might be better than the alternatives

I have been playing around with / working productively with AI in vim for a while now. For religious and practical reasons with local llms, but also with AI agents, namely kiro-cli, opencode and gemini cli, integrated into my workflow and environment with sidekick.

Through sidekick, the different AI agents were integrated in a separate buffer but with common and uniform keybindings, context integration (file content, diagnostics, ..) etc.

Still, every AI agent has its own interaction model, UI details, look and feel etc, and environment integration by sidekick, while impressive and mostly surprisingly stable, was rather hacky.

For my personal environment, I’ve been using gemini. My employer only provides kiro-cli and (limited) opencode, and I want to have an as-uniform experience as possible. So, for a while I would have wanted to have an abstraction over the cli agents in my environment.

Theoretically, that’s what ACP is for. However, ACP support in kiro-cli has not been present from the start, and I became rather familiar with the setup I had in the meantime.

Until I had a bit of time at my hands to touch the setup again last week.

The Problem

• I use nvim
• I (have to) use different AI agents (kiro-cli, opencode, gemini)
• I want a uniform look and feel, configuration, capabilities, etc. in my nvim environment, no matter which AI agent powers my AI workflow
• In most cases, the environment integration is more important to me than the agent specific capabilities and advantages that might be lost with it, especially when it comes to agent autonomy and orchestration

Possible solution

The Agent Client Protocol (ACP) standardizes communication between code editors/IDEs and coding agents.

If all used agents support the ACP, a single ACP client configuration could provide the solution I want.

Sadly, when I started the fun with “my” main agent kiro-cli, it has not supported ACP yet.

Solution

A while ago, kiro-cli added support for ACP 🥳!

No more reason not to try it out.

After a bit of research, I decided to use codecompanion to replace sidekick for the AI integration.

The full configuration can be found here, but I will write about a few details of the configuration.

Keybindings

I had the following sidekick keybindings:

A-a to open/toggle the agent window, including the selection as context, and A-i to toggle with the predefined prompt catalogue.

  keys = {
    {
      "<A-a>",
      function()
        require("sidekick.cli").toggle({ filter = { installed = true } })
      end,
      desc = "Sidekick Toggle",
      mode = { "n", "t", "i" },
    },
    {
      "<A-a>",
      function()
        require("sidekick.cli").send({
          filter = { installed = true },
          msg = "The current file is {file}, this is the current selection: \n\n```\n{selection}\n```",
        })
      end,
      desc = "Sidekick Toggle with context",
      mode = { "x" },
    },
    {
      "<A-i>",
      function()
        require("sidekick.cli").prompt({ filter = { installed = true } })
      end,
      desc = "Sidekick Select Prompt",
      mode = { "n", "t", "i", "x" },
    },
  },

For codecompanion, the same can be achieved with these keybindings:

      vim.keymap.set({ "n", "t", "i" }, "<A-a>", "<cmd>CodeCompanionChat Toggle<cr>", { desc = "CodeCompanion Toggle" })
      vim.keymap.set({ "x" }, "<A-a>", "<cmd>CodeCompanionChat Add<cr>", { desc = "CodeCompanion add selection" })
      vim.keymap.set(
        { "n", "t", "i", "x" },
        "<A-i>",
        "<cmd>CodeCompanionActions<cr>",
        { desc = "CodeCompanion Actions" }
      )

Buffer guard

The codecompanion buffer behaves like a normal buffer. Which means that if I jump in the jumplist while being in it, it would be replaced. Or if a custom / weird LSP action is triggered. Or…

So, I wanted to protect the codecompanion buffer, as identified through its virtual filetype.

init = function()
      -- If new buffers are opened or jumps being triggered while focus is on codecompanion, the action should be triggered outside of the codecompanion buffer
      vim.api.nvim_create_autocmd("FileType", {
        pattern = "codecompanion",
        callback = function(args)
          vim.bo[args.buf].buflisted = false
          local win = vim.fn.bufwinid(args.buf)
          if win == -1 then
            return
          end
          -- Guard this window: redirect any non-cc buffer that lands here
          vim.api.nvim_create_autocmd("BufWinEnter", {
            callback = function(ev)
              if not vim.api.nvim_win_is_valid(win) then
                return true
              end
              if vim.api.nvim_get_current_win() ~= win then
                return
              end
              if ev.buf == args.buf then
                return
              end
              -- Restore cc buffer in this window, move new buffer elsewhere
              vim.api.nvim_win_set_buf(win, args.buf)
              for _, w in ipairs(vim.api.nvim_tabpage_list_wins(0)) do
                if w ~= win and vim.bo[vim.api.nvim_win_get_buf(w)].filetype ~= "codecompanion" then
                  vim.api.nvim_win_set_buf(w, ev.buf)
                  vim.api.nvim_set_current_win(w)
                  return
                end
              end
              vim.cmd("split | buffer " .. ev.buf)
            end,
          })
        end,
      })
    end

tl;dr

• ACP is nice for Agent/Client integration
• setting up an work environment to use agent integration through ACP allows for the definition of common behaviour and user experience, even with different agents underneath it all
• nvim can be set up to use ACP with plugins, e.g. codecompanion
• ?
• profit

Note: Recently, pyrefly has gained the capabilities that were missing for me to be able to fully switch to it as sole lsp. My current setup doesn’t make use of cherry-picking LSP capabilities anymore.

As I have written about here, for most of my file edititing / programming / note-taking / … I use vim (neovim).

Not a bare-bones neovim, though. I use many plugins, and I have even more of them set up. Too many? Maybe. Probably. Especially considering the ever growing number of software supply chain attacks..

Anyways, that is a different topic for a different day.

Besides convenient plugins, the main building block for developer experience enhancement and efficiency are language servers, which provide stuff like diagnostics, code completion, navigation enhancements and more.

Besides playing around with its config, I try to actually use vim for productive work, including programming in python.

My go-to language server for python has been basedpyright, a fork of microsoft’s pyright, which basically adds the functionality of the proprietary pylance extension to pyright and adds some more bells and whistles.

While basedpyright is stable, helpful, convenient and generally excellent, there are a few new kids on the block:

There is a new generation of python language servers, written in Rust, targeting high performance and productivity in large and complex code bases:

There is Astral’s ty, Meta’s pyrefly and David Halter’s (Jedi’s original author) zuban, and probably some more.

I have been intrigued by them, mainly hoping for a smoother and more performant workflow in large and growing codebases. So, I have played around with them for a while now, eventually ending up using ty only for type checking while still having to rely on basedpyright for completion, auto-import, symbol renaming, navigation etc.

Why?

In single capabilities, none of the new lsp servers match basedpyright in my workflow. Yet.

E.g.:

• pyrefly has great completion, navigation and even auto-import, but it’s really bad in renaming symbols. It can rename variables, but no functions.
• ty has limited support of pydantic, which I use a lot, completion is not as powerful as pyrefly’s, but renaming works great.

Why not use both?

If you enable several language servers with the (nominally) same capabilities for the same file type, you get duplicate results/actions for every capability. Go to definition? Select where you want to jump to from these two identical alternatives!

My goal was to mix-and-match, e.g. use ty for renaming and pyrefly for everything else. However, while especially pyrefly’s configuration is very flexible, the lsps don’t support tailored enabling and disabling of select capabilities.

I tried for an unsuccessful while to configure the lsp server by restricting the handed-over client capabilities, but that didn’t go anywhere.

The only almost compromise that worked for me with the configuration through the lsp servers themselves was disabling basedpyright’s type checking and disabling ty’s language server capabilities, besides type checking. Like this, at least a little bit of the heavy lifting could be offloaded from basedpyright to ty, but I was not happy.

Now, I sort of am happy! I found out how to disable specific lsp capabilities for specific lsp servers through nvim’s lsp configuration. Instead of trying to get the lsp server to not provide the functionality, the client ignores it!

The Problem

I like working with Python, also with larger and growing code bases. There are LSP servers that offer excellent performance. However, none match the full functionality of basedpyright when used independently. Unfortunately, basedpyright lacks in performance. Combining multiple LSP servers introduces duplicate functionality and a bit of a mess. Such combination could be feasible if we could selectively disable or enable conflicting capabilities. However, it is not possible to instruct LSP servers to withhold certain functionalities unless they provide an interface for that, e.g. with dedicated configuration options.

The solution

Enable conflicting lsp servers, but tell the lsp client (nvim) that it should not use specific lsp server’s capabilities through an on_init hook 🥳!

The configuration is probably not fully final and will change, but this is the general thing.

  local lsp_capabilities = require("blink.cmp").get_lsp_capabilities(nil, true)

  ---@param client vim.lsp.Client
  local ty_on_init = function(client)
    client.server_capabilities.callHierarchyProvider = false
    client.server_capabilities.completionProvider = false
    client.server_capabilities.inlineCompletionProvider = false
    client.server_capabilities.inlineValueProvider = false
    client.server_capabilities.declarationProvider = false
    client.server_capabilities.definitionProvider = false
    client.server_capabilities.implementationProvider = false
    client.server_capabilities.inlayHintProvider = false
    client.server_capabilities.signatureHelpProvider = false
    client.server_capabilities.hoverProvider = false
    client.server_capabilities.referencesProvider = false
    client.server_capabilities.codeActionProvider = false
  end
  vim.lsp.config("ty", {
    capabilities = lsp_capabilities,
    on_init = { ty_on_init },
    settings = {
      ty = {
        disableLanguageServices = false,
        diagnosticMode = "workspace",
        experimental = {
          rename = true,
        },
      },
    },
  })

  ---@param client vim.lsp.Client
  local pyrefly_on_init = function(client)
    client.server_capabilities.renameProvider = false
  end
  vim.lsp.config("pyrefly", {
    capabilities = lsp_capabilities,
    on_init = { pyrefly_on_init },
    settings = {
      python = {
        pyrefly = {
          disableLanguageServices = false,
          displayTypeErrors = "force-on",
        },
        analysis = {
          diagnosticMode = "workspace",
          inlayHints = {
            callArgumentNames = "all",
            variableTypes = true,
            functionReturnTypes = true,
            pytestParameters = true,
          },
        },
      },
    },
  })

tl;dr

• Some lsp servers are not powerful enough to fully replace alternatives but would be great enhancements or replacements for single capabilities
• nvim’s lsp config API allows for specifying different hooks, on_init and on_attach etc.
• It’s possible to disable specific capabilities in those hooks

As I have written about here, I sort of have accepted a compromise in security for added convenience, auto-unlocking my LUKS encrypted root partition making use of my system’s built in TPM in order to only auto-unlock the system under specific (secure) conditions.

I’ve used clevis so far to manage the setup, using the trusty initramfs-tools to configure and bake the initramfs with the required modules and configuration, relying on cryptsetup to manage and reset the specifics of the configuration, e.g. whenever a bios update invalidated a PCR measurement.

Not that elegant. And there are new, more elegant and capable mechanisms available!

• There is systemd-cryptenroll to enroll physical tokens to LUKS2 encrypted volumes, simplifying the setup and management of this part of the process
• There is dracut for the configuration and management of initramfs images, the designated successor of initramfs-tools

While my previous setup worked, I wanted to try setting it up with the (designated) successor systems.

Well. To be honest, first I just wanted to try out systemd-cryptenroll instead of clevis, realized that the intended way of configuring auto-unlocking didn’t work with initramfs-tools, realized that (due to that, amongst other things) there have been discussions about replacing initramfs-tools with dracut as default and then decided to redo the whole process.

The Problem

Well, it’s not really a new problem, it’s an itch and a mental problem, I guess 😅. The original challenge, shifting the compromise of convenience vs. security a little by auto-unlocking the encrypted system with the help of the TPM has been working for a while.

But:

• systemd-cryptenroll exists, potential successor to clevis
• dracut exists, potential successor to initramfs-tools
• Both of those systems are very capable
• My setup is using their (potential) predecessors

The solution

systemd-cryptenroll

Installing systemd-cryptenroll on Debian is easy enough. It comes with systemd-cryptsetup, which might be on your system already? If not:

sudo apt-get install systemd-cryptsetup

Then, enrolling the TPM with the encrypted partition is pretty straightforward:

sudo systemd-cryptenroll --tpm2-device=auto /dev/nvme0n1p3

Depending on your needs/level-of-paranoia, you might want to add other PCRs, as systemd-cryptenroll per default only uses PCR 7, maybe you even want to use TPM + PIN:

sudo systemd-cryptenroll --tpm2-device=auto --tpm2-pcrs=7+11 --tpm2-with-pin=yes /dev/nvme0n1p3

You follow the guided process, and then can verify success. Hopefully.

sudo systemd-cryptenroll /dev/nvme0n1p3
SLOT TYPE
   0 password
   1 tpm2

Configuration should (™️) work by configuring the tpm in /etc/crypttab and rebuilding the initramfs with it:

cat /etc/crypttab
nvme0n1p3_crypt UUID=e010d6c1-6016-4f2d-bd05-1be8f383ada9 none tpm2-device=auto,luks,discard

building the initramfs with initramfs-tools (fail)

But, alas, rebuilding the initial ram disk fails on Debian and should also fail on Ubuntu:

sudo update-initramfs -u -k all
...
cryptsetup: WARNING: nvme0n1p3_crypt: ignoring unknown option 'tpm2-device'
...

After a bit of research: initramfs-tools doesn’t support tpm2 devices. There is a bug report and even an open merge request with a fix, but the situation appears to be stale.

However, there is the designated successor of initramfs-tools: dracut, which has replaced initramfs-tools in a bunch of linux distributions already.

building the initramfs with dracut

On debian and Ubuntu, it can be installed from the package sources like this, replacing initramfs-tools in the process:

sudo apt-get install dracut

Configuration of dracut happens in /etc/dracut.conf.d/*.conf.

I configured dracut to run the host-only initramfs creation with tpm2 support and compress with lz4, all in separate files:

cat /etc/dracut.conf.d/compression.conf
compress="lz4"

cat /etc/dracut.conf.d/hostonly.conf
hostonly="yes"
hostonly_cmdline="no"

cat /etc/dracut.conf.d/tpm.conf
add_dracutmodules+=" tpm2-tss crypt "
add_drivers+=" tpm tpm_tis_core tpm_tis tpm_crb "

With that configuration in place, the initramfs can be (re)built with

dracut -f

After that, a reboot should end up with an automatically unlocked encrypted partition.

tl;dr

If you trust your system firmware and tpm enough, you can set your system up to decrypt automatically without providing a passphrase using systemd-cryptenroll and dracut.

There are also other methods, but this one appears to be the most future proof and easy to set-up and maintain.

The Problem

My employer is a typical German large scale corporation. One of the attributes that come with that is that we are, of course, interested in generative AI, but also very sceptical and cautious.

Slowly, however, more and more AI services become available to us, but basically only web-interface chat solutions without APIs and IDE/development related integrations.

I was moderately happy when recently it was announced that we can use Amazon Q Developer! Awesome! Finally a development focused solution!

There are extensions for JetBrains IDEs, VS Code, Visual Studio and a preview for Eclipse, and there is a cli.

That should be enough, right?

…

Well. Not for all of us neovim users.

I want to have integration to the full potential of generative AI, adhering to applicable compliance restrictions, and still remain in the tool environment that makes me as productive and makes my work as enjoyable as possible.

Possible solution

There are many pretty excellent neovim AI plugins out there, e.g. Avante or Copilot Chat.

While those are pretty nice and powerful, most of them are taylored around a number of the big AI API providers, and are pretty big to work with.

Amazon Q does not provide a nice API, especially not an API that is compatible with existing clients and integrations. I spent a few minutes looking through Avante and did not see a quick path to extend the existing (web API centric) model adaption with Q.

I have been tinkering with a fork of neoai.nvim, extending it a little for my usecases, focussing on low-overhead integration of local language models with llama.cpp. I’ve been very happy with it, especially because the plugin is still so small that I don’t have a hard time understanding what it does and how to extend it without having to spend too much time. I’m lazy..

So, I decided to simply extend the plugin I’m most familiar with, my fork.

Solution

As written above, Amazon Q does not provide a nice API, but has a cli, called q. I decided to keep it simple and simply dump my prompt into q in non-interactive mode, parsing its stdout output. I hoped that it would allow me to use the existing way of providing persistent chat sessions and context, as well as having a simple way to apply existing functionality, e.g. code recognition for easy pasting with :put c etc.

It was surprisingly easy, the core of it all is fifty-ish lines of lua:

local utils = require("neoai.utils")

--- This model definition supports Amazon Q CLI integration
---@type ModelModule
local M = {}

M.name = "Amazon Q"

M._chunks = {}

M.get_current_output = function()
 return table.concat(M._chunks, "")
end

---@param chunk string
---@param on_stdout_chunk fun(chunk: string) Function to call whenever a stdout chunk occurs
M._recieve_chunk = function(chunk, on_stdout_chunk)
 -- Split the input chunk by newlines
 on_stdout_chunk(chunk)
 table.insert(M._chunks, chunk)
end

---@param chat_history ChatHistory
---@param on_stdout_chunk fun(chunk: string) Function to call whenever a stdout chunk occurs
---@param on_complete fun(err?: string, output?: string) Function to call when model has finished
M.send_to_model = function(chat_history, on_stdout_chunk, on_complete)
 -- Format messages for Amazon Q CLI
 local messages_json = vim.json.encode(chat_history.messages)

 chunks = {}

 local command = {}
 command = {
  "chat",
  "--no-interactive",
 }
 for _, v in ipairs(chat_history.params) do
  table.insert(command, v)
 end
 table.insert(command, messages_json)

 -- Execute the Amazon Q CLI command
 utils.exec("q", command, function(chunk)
  M._recieve_chunk(chunk, on_stdout_chunk)
 end, function(err, _)
  -- Clean up the temporary file

  on_complete(err, M.get_current_output())
 end)
end
return M

Q behaves a bit weird, it only marks source-code it produces with colors, using ansi codes, instead of using the de-facto standard backtics. Luckily it appears to be consistent, so code snippet extraction works like this:

extract_code_snippets = function(text)
  local matches = {}
  -- Amazon Q marks code by color-coding it green with ansi codes.
  -- Everything between "start green" and "reset color" is assumed to be code.
  for match in string.gmatch(text, "\27%[38;5;%d+m(.-)\27%[0m") do
    table.insert(matches, match)
  end
  return table.concat(matches, "\n\n")
end

I run environments that do not have q available, so I made my config sensitive to its availability.

config = function()
      local q_available = vim.fn.executable("q") == 1

      local models = {}
      local extract_code_snippets

      if q_available then
        extract_code_snippets = function(text)
          local matches = {}
          -- Amazon Q marks code by color-coding it green with ansi codes.
          -- Everything between "start green" and "reset color" is assumed to be code.
          for match in string.gmatch(text, "\27%[38;5;%d+m(.-)\27%[0m") do
            table.insert(matches, match)
          end
          return table.concat(matches, "\n\n")
        end

        table.insert(models, {
          name = "q",
          model = "q",
          params = { "--trust-all-tools" },
        })
      else
        startLlama()

        extract_code_snippets = function(text)
          local matches = {}
          for match in string.gmatch(text, "```%w*\n(.-)```") do
            table.insert(matches, match)
          end

          -- Next part matches any code snippets that are incomplete
          local count = select(2, string.gsub(text, "```", "```"))
          if count % 2 == 1 then
            local pattern = "```%w*\n([^`]-)$"
            local match = string.match(text, pattern)
            table.insert(matches, match)
          end
          return table.concat(matches, "\n\n")
        end

        table.insert(models, {
          name = "openai",
          model = "llama 3",
          params = nil,
        })
      end

      require("neoai").setup({ ---@diagnostic disable-line: missing-fields
        ui = {
          output_popup_text = "AI",
          input_popup_text = "Prompt",
          width = 45, -- As percentage eg. 45%
          output_popup_height = 80, -- As percentage eg. 80%
          submit = "<Enter>", -- Key binding to submit the prompt
        },
        models = models,
        register_output = {
          ["a"] = function(output)
            return output
          end,
          ["c"] = extract_code_snippets,
        },
        inject = {
          cutoff_width = 75,
        },
        prompts = {
          context_prompt = function(context)
            return "Please only follow instructions or answer to questions. Be concise. "
              .. (vim.api.nvim_buf_get_name(0) ~= "" and "This is my currently opened file: " .. vim.api.nvim_buf_get_name(
                0
              ) or "")
              .. "I'd like to provide some context for future "
              .. "messages. Here is the code/text that I want to refer "
              .. "to in our upcoming conversations:\n\n"
              .. context
          end,
          default_prompt = function()
            return "Please only follow instructions or answer to questions. Be concise. "
              .. (
                vim.api.nvim_buf_get_name(0) ~= ""
                  and "This is my currently opened file: " .. vim.api.nvim_buf_get_name(0)
                or ""
              )
          end,
        },
        mappings = {
          ["select_up"] = "<C-k>",
          ["select_down"] = "<C-j>",
        },
        open_ai = {
          url = "http://127.0.0.1:9741/v1/chat/completions",
          display_name = "llama.cpp",
          api_key = { ---@diagnostic disable-line: missing-fields
            value = nil,
            get = function()
              return ""
            end,
          },
        },
      })
    end,

Secret Agents

Reading through the config you might see a dangerous option: params = { "--trust-all-tools" },.

This allows q to execute all commands that are available to it in the current context.

Yes, it’s dangerous, probably also a bit stupid. But really powerful, as well. And a lot of fun, to be honest.

First I was annoyed by Amazon Q not providing a web API, but using the cli as API makes the tool integration agentic, basically for free!

I give the currently open file as context in the system prompt.

.. (
  vim.api.nvim_buf_get_name(0) ~= ""
    and "This is my currently opened file: " .. vim.api.nvim_buf_get_name(0)
  or ""
)

With this, I can refer to the current file and let q interact with it.

Or even the entire project!

I was pretty amazed by pressing <Alt-a> in my editor and watching the integration perform a full Please run the tests in this project and tell me if they were a success. and things like Create a feature branch hinting that I created a new UI, commit the latest changes and push them.

tl;dr

• Amazon Q is a developer centric AI by AWS
• Amazon Q does not provide a classic web API
• Amazon Q provides a cli
• The cli can be integrated in neovim
• The integration provides powerful assistance, through context-handover and chat response or by interacting with entire projects through shell command execution and interacting with files

The Problem

When working on several projects in different environments, interacting with distributed and relatively loosely coupled systems and platforms, such as different Jira, GitLab, … instances and other API providers, e.g. OpenAI or an Artifactory or whatever.

I want to be able to consume these systems from my development environment, which is basically my shell with dedicated command line interfaces and integrations in vim.

How can I now configure all the endpoints and other environmental aspects, including API tokens and other secrets, independently for different project contexts, without having to copy-paste secrets or hard-code secrets somewhere, e.g. putting clear-text tokens into .env files?

The solution should handle

• Automation and convenience. Whenever I enter a specific directory that specifies a context, e.g. a project or group of projects, I want the environment to automagically be set up correctly. When I leave the environment or change context, I want the environment to be teared down again.
• Centralization. Don’t repeat yourself (DRY) is not just a good practice for programming. I don’t want to have 27 places to touch when I changed my GitHub API token.
• Security. No clear-text secrets stored somewhere, ever.

Possible Solution

In pretty much every Linux environment that is not KDE, you should have the gnome-keyring available. The gnome-keyring is basically a local database that is encrypted with the user’s login password (-> automatically decrypted on login through pam), which provides a convenient centralised interface for secrets/keys. gnome-keyring supports gpg and ssh keys, which it can unlock and act as a key agent for, and generic passwords.

All passwords can then be retrieved by libsecret and the according tooling. There is a built-in git integration (built into git, but you still have to configure it!), and many applications use it already, e.g. the network manager, Chrome, Chromium, Edge, …

This opens a solution path, if it would be possible to automatically retrieve secrets from the gnome-keyring and create environment variables etc.

Solution

There is a cli, secret-tool, which allows you to store and retrieve generic secrets from/into the gnome-keyring. In combination with your shell setup through direnv, this allows for pretty nice and still secure automation, fulfilling all the goals.

Setup

gnome-keyring

Well. You need the gnome-keyring to be set up, ideally unlocked automatically on login. If you’re on gnome, that should come as a given. If you are using a different environment, it might be a given.

secret-tool

Install the secret-tool. On Debian based systems, which includes all the Ubuntu flavours, that’s done with

sudo apt-get install libsecret-tools

direnv

Install direnv, e.g., if you are on Debian based systems, with

sudo apt-get install direnv

And make sure to set it up and integrate it into your shell. I am using zsh, so you will find the configuration in the .zshrc

...
if command -v direnv 1>/dev/null 2>&1; then
  _evalcache direnv hook zsh
fi
...

How does it work?

For example, I want to have a gitlab API token in all my repositories for https://private-gitlab.com, so that I can quickly create new projects or open / interact with merge requests with the gitlab cli, glab. So, I have a file ~/workspace/private-gitlab.com/.envrc with the following content:

export GITLAB_TOKEN=$(secret-tool lookup protocol https server private-gitlab.com user <bob@secret.com>)
export GITLAB_URI="https://private-gitlab.com"
export GITHUB_TOKEN=$(secret-tool lookup protocol https server github.com user <dominik.gierlach@gmail.com>)

Together with direnv set up and integrated in to the shell, as can be seen im my dotfiles., this retrieves a token from the keyring whenever I enter a repository in ~/workspace/private-gitlab.com/... and sets it up as local environment variable in this context.

Add / change secrets

The secrets themselves can be set up like this:

secret-tool store --label='Git: <https://github.com/>' schema org.gnome.keyring.NetworkPassword protocol https user <dominik.gierlach@gmail.com> server github.com

Or by using seahorse, a graphical manager for the gnome-keyring.

Nested environments

With direnv, it’s possible to nest environments in order to extend them.

Assume a directory structure like this

~/workspace/github.com/..
  - .envrc
+ - project_group_1
  + - project_a
  + - project_b
+ - project_group_2
    - .envrc
  + - project_c
  + - project_d

I want to be able to use the environment configuration that is part of ~/workspace/github.com/.envrc in every project, but I want to change/extend it for all projects in ~/workspace/github.com/project_group_2.

That is possible by referring to the overarching .envrc from ~/workspace/github.com/project_group_1/.envrc:

source ~/workspace/github.com/.envrc
export AWS_PROFILE=PROJECT_GROUP_2_AWS

Bonus: Setup git credential integration with gnome-keyring

As written above, gnome-keyring can also store git credentials, when using https authentication. While this is somewhat orthogonal to setting up shell environments with secrets, it’s connected and important.

git can use credential helpers to store and automagically use credentials for different repositories. The trivial and built-in credential helper is store, which simply stores the credentials on disk in clear text, so let’s not go down that path.

The credential helper to integrate with gnome-keyring on linux is git-credential-libsecret.

Install git-credential-libsecret

On Debian based systems, including Ubuntu, the credential helper is not available as compiled binary, but the sourcecode is part of the git package.

Install it e.g. like this:

sudo apt-get install libsecret-1-dev
mkdir -p ~/workspace/other/libsecret
cd ~/workspace/other/libsecret
cp -r /usr/share/doc/git/contrib/credential/libsecret/* ./
make
mkdir -p ~/.local/bin
cp git-credential-libsecret ~/.local/bin

Set up git-credential-libsecret

After it has been installed, let’s make git use it.

git config --global credential.helper libsecret

Automatic setup with dotfiles and tuning

Yes, that was annoyingly complex, especially for a thing that should not be forgotten whenever a new machine is set up.

That’s why I integrated the setup into my dotfiles. I don’t want to have to remember (= look up and re-understand) how to set it up whenever I set up a new machine, and I don’t want to forget to set it up at all.

The git configuration is part of the global .gitconfig, installation is automated through tuning.

tl;dr

• Sometimes secrets are needed in (shell based) environments, to be set up as environment variables
• Tokens can be stored centrally and encrypted in the gnome-keyring
• Tokens can be retrieved and exported as environment variables with secret-tool
• Environment specific automation thereof can be done with direnv

Context

I am a big vim (neovim) fan and do basically all my programming and writing in it, the exception being work things that need MS Office. Luckily, most of those things are possible with the browser-based version of office, the exception being macros and obscure things like Powerpoint footer changes. Anyways, that is a completely different story.

Naked, vanilla vim is nice and present in basically every environment. So, it’s good to be able to use it, especially for remote configuration/documentation/troubleshooting work. The real power is unleashed with more advanced capabilities and extensions, which extend the barebone editor with modern creature comforts, such as linting, automatic fixing, code completion, auto formatting, syntax highlighting, test execution, debugging and intelligent navigation.

My code completion and intelligent navigation setup started with YouCompleteMe in 2017. I needed it mainly for C, and it did a good job.

A bit later, Microsoft’s Language Server Protocol gained much more traction, with more and more really usable language servers being really usable and taking off, mainly n the context of visual studio code. Most of the Language servers where installed and integrated into vscode through dedicated plugins.

In comes the neovim plugin coc (conquer of completion). This plugin acts as a language server client, meaning that it can use any language server that implements the lsp standard, and it can integrate slightly adapted vscode pluins as coc plugins. This meant that the managed installation of language servers and the integration thereof became really simple and straightforward, with a very active ecosystem and community around it. I migrated to coc in 2019, adding support and better integration for many more languages into my setup.

coc with a set of plugins and a handful of “manually” integrated language servers as well as with a set of linters and formatters, integrated with diagnostic-languageserver, was my default setup for 5 years. I changed the supported languages all the time, based on what I needed, but besides that and rewriting the config to lua, my setup stayed like this for 5 years.

In the meantime, neovim gained native language server client support, removing the need for a dedicated plugin for lsp support. However, coc still provided the easy integration and awesome out of the box functionality.

The awesome and very capable community embraced the feature, though. There is mason, an nvim native package/lsp manager, and many integration layer plugins between neovim and mason, making the integration of language servers slimmer (no extra layer), extremely flexible and manageable, with concise and powerful configuration.

Powered by all of that, projects like lazyvim started, providing much more out of the box functionality and ease of use than coc had. Lazyvim is basically a full-featured one size fits all preconfiguration of nvim, with very sane defaults and a smart plugin and configuration system that makes it feature rich without bloat. I know a few people who recently went down the rabbithole of vim, and they started with lazyvim right away. I respect it a lot, even though I want to have a bit more control over my setup, so not for me. The ecosystem grew and grew and is now more active and powerful than coc’s ecosystem.

I was very interested in the principle of the powerful and manageable modularization, but still, the normal out of the box experience was a lot worse than with coc, as it was basically not a box, but a collection of elements you had to build your box with yourself. The capabilities of coc are split into separate and independent components for completion (with lsp as source amongst others), dedicated components for formatters, linters, dedicated configuration of all components and system integration.

Interesting, but it’s a lot. Especially because my current setup just worked and did mostly what I wanted it to do, and I had no real functioning reason to migrate besides the itch. So, I put a migration on the back of my to do list, to do some other time.

Some other time is now

That is until for the first time ever I had to do things in C#. There is a coc plugin for C#, coc-omnisharp, which in the README greets you with the message:

⛔ This project is lacking proper maintanence. I would recommend csharp-ls at this moment.

This leaves manual integration of the csharp-language-server. Awesome, I did it right away.

But there was a problem.

With the coc integration, I was unable to find a way to replace the default sync handler for system libraries only available as binaries, used e.g. for navigation, such as go_to_definition, with a buffer showing the decompiled sourcecode. This is explicitly (and easily) possible with the flexible configuration of native language servers in nvim, as documented.

So, last weekend, I did it.

I migrated my configuration.

Structure

I use lazy as plugin manager, which makes it really simple to split up plugin configuration. I decided to split my language server config into a dedicated file for the abstract definition of the behaviour of lsps, e.g. with key mappings, sources of the completion system and so on, a dedicated file for the specific definition of supported languages, and a dedicated file for the separate configuration of linters and fixers, while also revamping the test execution and debugging setup.

/home/gierdo/.dotfiles/.config/nvim/lua/plugins

❯ tree
.
├── debugging.lua
├── filetree.lua
├── fuzzyness.lua
├── git.lua
├── init.lua
├── linters.lua
├── lsp-behaviour.lua
├── lsp-languages.lua
├── neoai.lua
├── testing.lua
├── theme.lua
└── treesitter.lua

Copy-pasting the few hundred lines of lua here probably makes this post even less readable, but I’ll point out some things here that are not just obvious configuration.

If you want to see the full configuration, check out my dotfiles.

The original motivation: C# decompiler integration

      local cs_config = {
        capabilities = lsp_capabilities,
        handlers = {
          ["textDocument/definition"] = require("csharpls_extended").handler,
          ["textDocument/typeDefinition"] = require("csharpls_extended").handler,
        },
      }
      lspconfig.csharp_ls.setup(cs_config)

Fuzzy and fancy selection for multiple navigation target hits

          vim.keymap.set(
            "n",
            "gd",
            require('telescope.builtin').lsp_definitions,
            { buffer = event.buf, desc = "Go to definition." }
          )
          vim.keymap.set(
            "n",
            "gi",
            require('telescope.builtin').lsp_implementations,
            { buffer = event.buf, desc = "Go to implementation." }
          )
          vim.keymap.set(
            "n",
            "go",
            require('telescope.builtin').lsp_type_definitions,
            { buffer = event.buf, desc = "Go to type definition." }
          )
          vim.keymap.set(
            "n",
            "gr",
            require('telescope.builtin').lsp_references,
            { buffer = event.buf, desc = "Show references." }
          )

Extended json- and yaml schema support

      lspconfig.jsonls.setup({
        capabilities = lsp_capabilities,
        settings = {
          json = {
            schemas = require("schemastore").json.schemas(),
            validate = { enable = true },
            schemaDownload = { enable = false },
          },
        },
      })

      lspconfig.yamlls.setup({
        capabilities = lsp_capabilities,
        settings = {
          yaml = {
            schemaStore = {
              enable = false,
              url = "",
            },
            schemas = require("schemastore").yaml.schemas({
              extra = {
                {
                  name = "Cloudformation",
                  description = "Cloudformation Template",
                  fileMatch = { "*.template.y*ml", "*-template.y*ml" },
                  url = "https://raw.githubusercontent.com/awslabs/goformation/master/schema/cloudformation.schema.json",
                },
              },
            }),
            customTags = {
              -- Cloudformation tags
              "!And scalar",
              "!If scalar",
              "!Not",
              "!Equals scalar",
              "!Or scalar",
              "!FindInMap scalar",
              "!Base64",
              "!Cidr",
              "!Ref",
              "!Sub",
              "!GetAtt sequence",
              "!GetAZs",
              "!ImportValue sequence",
              "!Select sequence",
              "!Split sequence",
              "!Join sequence",
            },
          },
        },
      })

imho smart test execution and debugging keymaps

      local wk = require("which-key")

      wk.add({
        { "<leader>t", group = "   Test" },
        { "<leader>tr", neotest.run.run, desc = "Run nearest test." },
        {
          "<leader>tf",
          function()
            neotest.run.run(vim.fn.expand("%"))
          end,
          desc = "Run current file.",
        },
        {
          "<leader>td",
          function()
            neotest.run.run({ strategy = "dap" })
          end,
          desc = "Debug nearest test.",
        },
        {
          "<leader>ts",
          neotest.summary.toggle,
          desc = "Show/hide summary.",
        },
      })

      wk.add({
        { "<leader>d", group = "   Debug" },
        {
          "<leader>dd",
          telescope.extensions.dap.commands,
          desc = "Show debug commands.",
        },
        {
          "<leader>dt",
          dap.toggle_breakpoint,
          desc = "Toggle line breakpoint on the current line.",
        },
        {
          "<leader>dbs",
          dap.set_breakpoint,
          desc = "Set breakpoint.",
        },
        {
          "<leader>dbl",
          telescope.extensions.dap.list_breakpoints,
          desc = "List breakpoints.",
        },
        {
          "<leader>dv",
          telescope.extensions.dap.variables,
          desc = "Show variables.",
        },
        {
          "<leader>dbc",
          dap.clear_breakpoints,
          desc = "Clear breakpoints.",
        },
        {
          "<leader>dbe",
          dap.set_exception_breakpoints,
          desc = "Set Exception breakpoints.",
        },
        {
          "<leader>dc",
          dap.continue,
          desc = "When debugging, continue. Otherwise start debugging.",
        },
        {
          "<leader>dfd",
          dap.down,
          desc = "Move down a frame in the current call stack.",
        },
        {
          "<leader>dfu",
          dap.up,
          desc = "Move up a frame in the current call stack.",
        },
        { "<leader>dp", dap.pause, desc = "Pause debuggee." },
        {
          "<leader>dr",
          dap.run_to_cursor,
          desc = "Continues execution to the current cursor.",
        },
        {
          "<leader>dre",
          dap.restart,
          desc = "Restart debugging with the same configuration.",
        },
      })

tl;dr

• language servers and such + vim are great
• there are many ways to integrate vim with language servers
• coc is not that slim and flexible with good out of the box experience
• native nvim lsp support is slim and flexible with worse out of the box experience
• with existing plugins and thanks to the community, benefiting from the advantages of native nvim lsp support is possible in a sane way

I’m a big fan of atuin, a shell history manager, which replaces the flat command history file with a local database, fully searcheable and with builtin synchronization support.

My setup of atuin, installation and configuration, can be seen in my dotfiles. But the documentation is excellent.

In case you are wondering why the sync configuration is not in my config: I try to keep all specific configuration of stuff that is not publicly available and might not be needed in all my environments outside of the dotfiles. I set $ATUIN_SYNC_ADDRESS in my ~/.profile.local.

export ATUIN_SYNC_ADDRESS=http://raspberrypi.local:8888

Atuin offers a free synchronization server at https://api.atuin.sh. Yes, it’s not a good idea to show and open up your command history to everybody. Which is why the synchronization is completely end-to-end encrypted by atuin. The synchronization server only sees gibberish, and all your clients have to be set up with a passkey to enable sync.

If you, like me, still don’t want to trust that data to a sync service, you can also run your own synchronization server, which is pretty straightforward, assuming you know a bit about services and postgresql.

Of course that means that you have to run a server somewhere…

I happen to have a rasperry pi 3 in my home network, which mainly acts as an always-on member of syncthing-synchronized directories, and as a git repository server for private repositories with gitolite. I don’t expose the Pi at all to the public internet, it is only available internally, resolveable through mDNS. I wanted to set it up as atuin synchronization server. It’s already there, it is available for all devices in my local home network. This, of course, means that I can only synchronize shell history with my machines at home. But I don’t have a problem with this limitation at all.

The problem

Atuin is not installable through the raspbian package sources. This is also true for Debian at the time of writing. On my machine, I install it with cargo, orchestrated with tuning.

So, how do you install atuin on my raspberry pi, which uses an ARM CPU? Especially if you don’t want to compile it on the pi itself, wait for ages and fill your pi with stuff you don’t really need?

The solution

Atuin is a program written in Rust. This means that

1. Programs are compiled to architecture specific binaries
2. Programs can be built pretty easily with Rust’s awesome package manager cargo
3. Cross compilation is pretty straightforward, thanks to Rust.

As usual, the devil is in the details, however.

The details

Setting up Rust for cross-compilation

With rustup as rust toolchain manager, it’s easy to install the rust specific target and toolchain for the raspberry pi.

$ rustup target add armv7-unknown-linux-gnueabihf

This is not enough, however. For successful cross compilation, rust also needs the target specific gcc toolchain to link against C libraries and sysroot.

Setting up the system

You need gcc for cross-compilation with the armhf target, as well as the armhf specific libraries for dynamic linking.

On a Debian system, this can be installed like so.

$ sudo apt-get install gcc-arm-linux-gnueabihf libgcc-13-dev-armhf-cross

The installation of gcc and the sysroot is on a system level, independent from the local rust setup. This means that cargo has to be set up correctly so that it uses the correct linker.

Configuring cargo

Target specific attributes can be configured in ~/.cargo/config.toml

[target.armv7-unknown-linux-gnueabihf]
linker = "arm-linux-gnueabihf-gcc"

Compilation

After everything has been set up, cargo can be built in the local directory with this command:

$ cargo install --target=armv7-unknown-linux-gnueabihf --target-dir=$(pwd) --root=$(pwd) atuin

This will download the sources of atuin and all dependencies, cross-compile it and copy it to a bin subdirectory.

$ readelf -h bin/atuin
ELF Header:
  Magic:   7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00
  Class:                             ELF32
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            UNIX - System V
  ABI Version:                       0
  Type:                              DYN (Position-Independent Executable file)
  Machine:                           ARM
  Version:                           0x1
  Entry point address:               0xd363d
  Start of program headers:          52 (bytes into file)
  Start of section headers:          32358908 (bytes into file)
  Flags:                             0x5000400, Version5 EABI, hard-float ABI
  Size of this header:               52 (bytes)
  Size of program headers:           32 (bytes)
  Number of program headers:         10
  Size of section headers:           40 (bytes)
  Number of section headers:         44
  Section header string table index: 43

Installation

Copy the binary onto your raspberry pi, e.g. to /usr/local/bin/atuin, and you’re good to go!

tl;dr

• atuin is great to manage shell history
• atuin sync is great and pretty safe
• atuin sync on a raspberry pi under your control is even safer
• cross-compilation with Rust is relatively easy
• the devil is in the details

You might have read my previous posts about using a local codellama for a vim coding assistant, setting it up to use my laptop’s GPU for inference and the performance evaluation and so on.

Also, my previous evaluation of different models, which formed the base of my selection of which model to run locally.

Since quite a bit of time, there has been a new kid on the block: Meta’s llama 3 is here! I’ve been running it since April, and it basically replaced my previous local model.

I will not repeat the evaluation of the previous models, but apply the same methods to llama 3. For the others, please take a look at the previous article. Or, even better, try them out yourself.

Environment

The general setup is described already in previous articles, mainly the performance evaluation of GPU assisted inference

Here is the short summary:

• Lenovo Thinkpad t14s gen 4
• llama.cpp with ROCm on AMD Ryzen 7 PRO 7840U w/ Radeon 780M Graphics, 8G VRAM

This results an boundary conditions for the models to chose from: The models have to be usable with llama.cpp and cope with 8GB of VRAM.

llama 3 fits that requirement nicely in the quanticised 8B instruct variant: Meta-Llama-3-8B-Instruct.Q4_K_M.gguf

Method

I gave each model a specific task:

Please create a c function for an embedded platform (no stl or time.h, no
POSIX), which returns the unix timestamp for a given time.

With the created code in the context, I want to test each model’s ability to work with given code, e.g. for comments and improvement.

What is happening here?

What are potential problems with the given code?

Please give me a well-documented version of this code. Please follow best
practices and do not comment overly verbosely.

llama 3

Llama 3 happy starts right away, ignoring significant parts of the instruction, however: The given value is not unix timestamp compliant.

--------

Here is a simple C function that calculates the Unix timestamp (seconds since
January 1, 1970, 00:00:00 UTC) from a given year, month, day, hour, minute and
second:

```c
#include <stdint.h>

uint32_t unix_timestamp(int year, int month, int day, int hour, int min, int sec) {
    uint32_t timestamp = 0;

    // Calculate the seconds since January 1, 1970, 00:00:00 UTC
    timestamp += (year - 1970) * 31536000; // 31536000 is the number of seconds in a year

    // Adjust for leap years
    if ((year % 4 == 0 && year % 100 != 0) || year % 400 == 0)
        timestamp++; // Add one second for February 29th

    int month_days[] = {31, 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31};
    timestamp += (month - 1) * 2628000 + day * 86400; // Add days and hours
    timestamp += (hour * 3600); // Add hours
    timestamp += (min * 60); // Add minutes
    timestamp += sec;

    return timestamp;
}
```

Note: This function assumes that the input values are valid. It does not
perform any error checking or handling.

--------

The good thing is that llama is aware of it when asked:

--------

Are you sure this function returns a unix compliant timestamp?

--------

I made a mistake. The function I provided earlier does not correctly calculate
the Unix timestamp.

A Unix timestamp is the number of seconds that have elapsed since January 1,
1970, at midnight in Coordinated Universal Time (UTC). To correctly calculate
this, we need to consider the following:

* The epoch date (January 1, 1970)
* The leap years
* The daylight saving time

Here's a revised version of the function that calculates the Unix timestamp
more accurately:
```c
#include <stdint.h>

uint32_t unix_timestamp(int y, int m, int d, int h, int mi, int s) {
    uint32_t timestamp = 0;

    // Calculate the seconds since January 1, 1970, 00:00:00 UTC
    timestamp += (y - 1970) * 31536000; // 31536000 is the number of seconds in a year

    // Adjust for leap years
    if ((y % 4 == 0 && y % 100 != 0) || y % 400 == 0)
        timestamp++; // Add one second for February 29th

    int month_days[] = {31, 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31};
    timestamp += (m - 1) * 2628000 + d * 86400; // Add days and hours
    timestamp += h * 3600; // Add hours
    timestamp += mi * 60; // Add minutes
    timestamp += s;

    return timestamp;
}
```
This function takes into account the leap years, but it still does not consider
daylight saving time. If you need to account for daylight saving time, you
would need a more complex algorithm.

Keep in mind that this function is still not fully compliant with the Unix
timestamp standard, as it does not take into account the start and end times of
daylight saving time.

--------

Now, this is pretty nice. And we are all just humans, after all.

Now, let’s see if it understands what it has created.

With the (revised) generated function in the context, let’s ask it what is happening, and what potential issues could be:

What is happening here?

--------

This is a C function named `unix_timestamp` that takes six integer arguments:
year (`y`), month (`m`), day (`d`), hour (`h`), minute (`mi`), and second
(`s`). It calculates the Unix timestamp (number of seconds since January 1,
1970, 00:00:00 UTC) based on these input values.

--------

What are potential problems with the given code?

--------

1. The function does not handle invalid inputs such as dates that do not exist
   (e.g., February 30, 29 or April 31), negative months, or times outside of
   the range 0-23 for hours and 0-59 for minutes.

2. It does not consider daylight saving time (DST) changes, which can affect
   the accuracy of the timestamp calculation.

3. The function assumes that the system's clock is set to UTC, but if the
   system is in a different timezone, the calculated timestamp will be off by
   the difference between the local timezone and UTC.

4. It does not handle overflow when calculating the timestamp. If the sum of
   seconds exceeds 2^32-1 (the maximum value for a uint32_t), the calculation
   will wrap around, resulting in an incorrect timestamp.

5. The function can be improved to use more descriptive variable names instead
   of single-letter variable names like 'y', 'm', 'd', 'h', 'mi', 's'.

6. It does not consider the fact that month days are different for February and
   it's not correct to subtract 1 from month, because January is also a month.

7. The function should return a time_t type instead of uint32_t.

8. The code can be simplified by removing the conditional statements in the
   calculation of leap year.

9. The variable `month_days` is not necessary and can be removed.

10. The comment for the line `timestamp  += (y - 1970) * 31536000;` should be
    updated to accurately describe what it does.

---------

Can you give me an improved version?

--------

Here is a revised version of your code:

```c
#include <stdint.h>

time_t unix_timestamp(int year, int month, int day, int hour, int minute, int second) {
    if (month < 1 || month > 12)
        return -1;

    time_t timestamp = 0;

    // Calculate the seconds since January 1, 1970
    int total_seconds_in_year = 31536000; // 31536000 is the number of seconds in a year
    if ((year % 4 == 0 && year % 100 != 0) || year % 400 == 0)
        ++total_seconds_in_year;

    int[] month_days = {31,28,31,30,31,30,31,31,30,31,30,31};
    int day_of_year = 0;
    for(int i = 1; i < month; i++) {
        day_of_year += month_days[i-1];
    }
    day_of_year += day - 1;

    timestamp = (year - 1970) * total_seconds_in_year + day_of_year * 86400 + hour * 3600 + minute * 60 + second;

    return timestamp;
}
```

This version includes error checking for the month, adds a comment to describe what each section of code does, and simplifies some calculations.

--------

Please give me a well-documented version of this code. Please follow best
practices and do not comment overly verbosely.


--------

Here is the code with clear comments:

```c
#include <stdint.h>

// Convert a date in the format (year, month, day, hour, minute, second) to a Unix timestamp.
time_t unix_timestamp(int year, int month, int day, int hour, int minute, int second) {
    // Check if the month is within valid range (1-12)
    if (month < 1 || month > 12) {
        return -1; // Return an error
    }

    time_t timestamp = 0;

    // Calculate the number of seconds in a year, taking into account leap years
    int total_seconds_in_year = 31536000;
    if ((year % 4 == 0 && year % 100 != 0) || year % 400 == 0) {
        ++total_seconds_in_year; // Leap year adjustment
    }

    // Array of days in each month (excluding February)
    int month_days[] = {31, 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31};
    int day_of_year = 0;

    // Calculate the total number of days before the given month
    for (int i = 1; i < month; i++) {
        day_of_year += month_days[i - 1];
    }
    day_of_year += day - 1; // Adjust for 0-based indexing

    // Calculate the Unix timestamp
    timestamp = (year - 1970) * total_seconds_in_year + day_of_year * 86400 + hour * 3600 + minute * 60 + second;

    return timestamp;
}
```

--------

Yes! nice. It also tries to improve it’s code.

Verdict

Llama 3 understands the given task but is lacking a bit of attention to detail. However, it can improve upon itsolf on reflection.

General non-coding tasks

Ignore the content of the given task, it is merely a repetition of the task from the previous evaluation.

Method

I put the following bullet points in the context of each model, and asked the modell to write the given points in full sentences for a short tech article.

• mistral is okay with coding tasks
• mistral appears to be good at general language tasks
• codellama appears to be good at coding tasks, with good knowledge of libraries
• codellama appears to be lacking at general language tasks

Please write the given points in full sentences for a short tech article.


--------

Here is the rewritten text:

Mistral is capable and competent when it comes to tackling coding tasks.
Additionally, Mistral demonstrates proficiency in general language-related
tasks. Codellama, on the other hand, excels at coding tasks, showcasing its
strong understanding of various libraries. However, Codellama's abilities
dwindle when it comes to general language tasks, indicating a specific area
where it requires improvement.

--------

Please elaborate a bit more.

--------

Here is an elaboration on the text:

Mistral's strengths lie in its ability to tackle coding tasks with ease and
proficiency. Whether it's working on a complex algorithm or debugging a piece
of code, Mistral is well-equipped to handle a wide range of programming
challenges.

In addition to its coding prowess, Mistral also demonstrates a strong
foundation in general language skills. It can understand and communicate
effectively through written text, making it an asset in any team that values
clear communication.

Codellama, on the other hand, stands out for its exceptional abilities in
coding tasks. Its deep understanding of various programming libraries and
frameworks allows it to create efficient, effective, and scalable solutions.

However, Codellama's strengths are largely limited to coding-related areas.
When it comes to general language tasks, such as writing coherent sentences or
conversing naturally, Codellama struggles. This highlights an area where it
requires improvement, potentially through exposure to more diverse linguistic
contexts and training exercises.

Overall, Mistral's versatility in both coding and language skills makes it a
valuable asset, while Codellama's specialized expertise in coding is noteworthy
but may benefit from further development in other areas.

--------

Summary

The expanded text, especially the elaborative version, is pretty much spot-on with the given information.

Verdict

I switched to llama. It yielded very nice results, while (this is not covered in this particular article, maybe I’ll write another one?) being very performant and being able to handle a relatively large context window locally in my environment, which is a large contributing factor to the decision.