GitHub Copilot Became An Artist

This article is part of the XPRT. Magazine #21

A bit of context

In the edition #18 of this magazine you can find my article about how I replaced manual 3D modeling with a code-driven workflow — building a p5.js torus knot renderer backed by an LLM workflow that goes from a text prompt to a plotter-ready configuration. The short version: I'm both a developer and always loved art, and that project was where those two sides finally properly met.

Building that tool is what pulled me into generative art properly. Not just visuals on screen — physical things. Lines on paper. Acrylic on canvas. The plotter doesn't forgive vague geometry.

After writing about it I thought I'd take a break and just play. No thesis, no plan. I opened a p5.js editor in my browser and started poking at different algorithms and creating sketches. Some I worked through on my own. On others I had Copilot as a thinking partner — describe what I wanted, get a structural proposal, refine together. Most of them eventually ended up on the plotter.

You can guess from the fact that I'm writing this that things went somewhere interesting. Where the experiments led, what Copilot's role was in getting there, and why I decided to go a little further and build a proper assistant — with agent files, reusable prompts, and custom instructions — is what this article is about.

The sketches — what I actually made

Let me walk you through some of the results I liked and used them in the future as references, because the variety is kind of the whole point.

Metaball - the distortion field (16 Feb 2026)

Here I was playing with order and chaos as a concept. From the same sketch I generated a few variations and what you see on the picture are tree real art pieces made with acrylic paint on canvas. An ordered grid covers the canvas — structured, almost mechanical. Accent was on metaballs: the blobs are filled in, solid forms pushing against the surrounding geometry. The contrast is the whole point.

Mint — wire graph (14 Feb 2026)

Metaball algorithm again — the math that makes blobs merge in liquid simulations — but instead of filling the blobs, I sample points inside the field and connect them into a wire graph via nearest-neighbour lookup.

Sakura flow (18 Feb 2026)

Hub-based node network with Perlin flow trails drifting across it. This sketch is one of my favourite, so it ended up having a few variations.

Tangled web (18 Feb 2026)

The most procedurally complex one. Random line candidates thrown at the canvas, intersections found, new edges added at each crossing, the whole graph physically relaxed until it settles. Three overlapping layers. It looks hand-drawn despite being completely algorithmic. I never got to the point of creating this one with the plotter machine just because it's not finished for me. I need to work more on this to fit my idea here.

Why build an assistant?

Variety was what hooked me. Every sketch came out different — sometimes shockingly so. Press R to cycle seeds, get three hundred variations, maybe two worth printing. That unpredictability is the whole appeal.

But it's slow. Understanding an algorithm well enough to tune it takes hours. And every new sketch starts with the same boilerplate: the CFG block, the regenerate() function, the keyboard shortcuts, the SVG export, the spatial hash. None of it is hard — it's just friction, right when you want to be in exploration mode.

If you're familiar with things like Genuary, you might recognise the dynamic. The process isn't linear. You're chasing new shapes without always knowing what you're looking for. Sometimes I had a specific equation in mind. More often I was just describing a feeling to Copilot and we'd navigate toward something together — when it worked it was genuinely exciting, and I'd stare at the code afterward trying to understand how we got there. When it didn't work, after a long session with good prompts, Copilot would just produce something wrong. That cycle of good sessions and disappointing ones is what eventually pushed me toward building something more durable.

The timing was right because I had enough material. Looking across the folder I could see common structures, shared utilities, recurring parameter shapes. That's what you need to give an assistant real context — not a list of rules but actual examples. Without the existing sketches I wouldn't have tried this. There's nothing to extract patterns from if there's nothing made.

So I started building: a workspace instruction file, a library of reusable prompts, two custom agents. One to make the art. One to write about it.

The plan — what we're building

Here's the practical setup. Inside the workspace there are now three things that didn't exist before:

.github/copilot-instructions.md — A workspace-level instruction file that Copilot reads automatically whenever you open the project. It contains: the project context (p5.js, plotter output, global mode), the non-negotiable structural conventions (CFG block, regenerate/draw split, keyboard shortcuts, SVG export format), plotter requirements (no fills, stroke weight ranges, line density guidelines), the aesthetic vocabulary (what visual qualities to aim for), and the full library of algorithms already implemented with their key parameters.

This file does something important: it means I never have to re-explain my workflow to Copilot. Every new conversation starts with Copilot already knowing that I work for a plotter, that LINESSVG needs to mirror every line, that the spatial hash is the standard utility for graph work. That context carries.

.github/prompts/ — Six individual .prompt.md files, each one a self-contained reusable prompt with a YAML frontmatter header. This is actually a feature of GitHub Copilot: prompt files are first-class objects the editor understands, not just text notes. Each file has an agent field pointing to the Artist agent and a description that appears in the Copilot UI. The six prompts cover:

File - What it does
01-new-sketch - Discover → propose → generate a new sketch through a short conversation
02-layer-algorithms - Compose two algorithms into one unified piece
03-debug-explain - Audit and explain what a sketch is doing and why
04-add-element - Add a new visual layer without touching existing code
05-explore-algorithm - Propose and implement an algorithm not yet in the workspace
06-intensify-density - Increase line density and visual intensity without changing the algorithm

All six prompts route to the Artist agent, which handles the distinction internally — asking questions and proposing before generating when that's the right phase, or going straight to code when the intent is already clear. The prompt is the starting gun; the agent decides the pace.

When a session produces something worth documenting, the Artist doesn't write it up directly. It hands off to the Writer agent — a structured brief, five fields, and then the Writer takes over. Separation of concerns: the Artist stays in code and parameter space, the Writer stays in the document. Consistency comes from neither agent trying to do both jobs at once.

A new sketch, step by step — lichen field

Here's where the theory turns into practice. The first experiment with the full setup in place: reaction-diffusion with a "chaotic growth" mood. An algorithm not yet in the workspace, a completely blank file.

One thing worth noting about how this session worked: I was actively documenting everything alongside Copilot as it happened — every decision, obstacle, and moment of surprise got logged. That's what feeds the Writer agent and eventually this article. The process writes itself in real time rather than being reconstructed afterward.

Here is how the final resul looked like, well one of the variations:

Why reaction-diffusion?

Reaction-diffusion wasn't in the original algorithm library. It's the system behind biological patterns — spots on a cheetah, stripes on a zebrafish, the branching of a leaf vein. Two chemicals diffusing and reacting with each other. Tune the parameters and you get spots, stripes, coral-like colonies, labyrinthine mazes.

The challenge for the plotter is that it produces a field of floating-point values, not lines. The solution is marching squares: walk across the grid cell by cell, find where the field crosses a threshold, emit a short line segment. Chain enough segments and you have the outline of the pattern. Multiple thresholds give layered nested contours — thousands of short line segments, perfect for the plotter.

What the instruction file actually did

Using 01-new-sketch.prompt.md as the starting point, the request was essentially: reaction-diffusion, chaotic growth, dark background, acid yellow-green accent. Because the instruction file was already loaded, the CFG convention, the regenerate()/draw() split, the LINESSVG export format — all of it came out correctly on the first pass. No boilerplate re-explaining.

What needed a manual fix: one missing + operator in the Laplacian computation. A copy/paste slip in dense arithmetic. Small, but a good reminder that generated numerical code always deserves a review pass.

Iteration: making it denser

The first result looked too sparse for a plotter print. The question became: what are the right levers? For reaction-diffusion there are four distinct ways to get more lines — switch the Gray-Scott regime, add more iso-threshold levels, run more iterations, spread seeds more widely across the canvas. This session produced 06-intensify-density.prompt.md: a reusable prompt that documents exactly those levers for each algorithm type, because what "add more lines" means is completely different depending on what algorithm you're in.

The prompt redesign

After building the lichen sketch, I looked back at 01-new-sketch.prompt.md and realised it didn't match what had actually worked. The original was a passive fill-in template that jumped straight to code. What worked in practice was a three-phase conversation: propose algorithm options first, lay out a plan in words before writing any code, then generate with explicit review notes. The prompt got redesigned to match reality.

The first version of a prompt is usually what you think you want. The useful version comes from noticing what actually worked and writing that down instead.

When the algorithm surprises you

Here's something that happened that I want to record because it's very much part of the process.

While pressing R to cycle through random seeds on the lichen sketch, one came out with a large natural void — an irregular empty region surrounded by tightly-wound contour lines. The pattern had grown around it organically, the iso-contours curving and narrowing as they framed the space. It looked intentional. It looked like a composition decision. It wasn't — it was randomness.

That moment is one of the things I love most about generative art: the algorithm produces something better than what you would have asked for. Creating an empty area on the sketch that is made by complex algorithms is not a trivial task, at least not for me, so seeing this made me very happy. Meaning that my parametrisation will allow me to have certain unpredictability in results.

Once you understand the simulation, making a deliberate void is easier. The B-chemical needs seeds to start growing — if you clear a circular region after seeding, no colony forms there, and the surrounding pattern grows around it. The iso-contours naturally frame the empty space. The opposite — a concentration accent — packs extra seeds tight in one region, creating a hotspot that develops the most complex internal structure in the composition.

This is the third time in this project that the most useful thing to add came from watching the algorithm rather than planning it.

Skills vs. prompts — a distinction worth making

After building out the prompt library, the next question was whether some of this knowledge should live somewhere else entirely. Specifically: GitHub Copilot Agent Skills.

Here's the distinction in plain terms.

Prompts are action requests. They're single-use per session, task-shaped, and invoked when you want something done right now. "Generate a new sketch." "Intensify the density." "Explain how this sketch works." Everything in the prompts/ folder is like this.

Skills are capability packages. They're knowledge-shaped, persistent, and they can bundle supporting assets alongside the instructions — reference tables, templates, data files, anything the instructions might need to point to. The key difference is not the format, it's what they contain and why you reach for them. You invoke a skill to make Copilot know something domain-specific; you invoke a prompt to make it do something.

This distinction matters because putting everything into prompts has a ceiling. A prompt like "generate a new sketch" can tell Copilot how to do it. But a skill for "this is what a plotter sketch core looks like" can bundle the actual CFG template and SVG export template as files — so the model doesn't reconstruct them from description, it reads the actual canonical version.

The clearest example: the Gray-Scott regime reference. While working on the lichen sketch we discovered hands-on that feed 0.037/kill 0.060 produces coral, 0.029/0.057 produces dense labyrinthine channels, and 0.025/0.055 produces fine intricate networks. That's real knowledge from real experimentation. A prompt can't hold a structured reference table effectively. A skill with a bundled regime-reference.md file can — and the next session working on a reaction-diffusion sketch starts with all of that already loaded, not re-discovered.

Two skills, one clear rule

Two skills were created:

gray-scott-plotter — The most domain-specific one and the most immediately valuable. The SKILL.md covers the algorithm, the iso-contour strategy, the density-intensification levers, and plotter safety rules. The bundled regime-reference.md is a full feed/kill parameter map — every regime named, described, and cross-referenced with iteration requirements and plotter implications.

p5js-algorithm-library — A living reference of all six algorithms currently in the workspace, with character descriptions, key parameters, plotter characteristics, and pairing suggestions. The bundled algorithm-params.md has full parameter tables with conservative/standard/dense presets for each algorithm — much more useful than the brief table in copilot-instructions.md.

The rule that emerged for deciding what goes where: instructions file for always-on conventions, prompts for things you invoke to do a task, skills for domain knowledge that benefits from bundled reference material. When the content is a table, a template, or a parameter map that you'd want to look up rather than reconstruct, it belongs in a skill asset — not in a markdown list inside an instructions file.

What this means in practice is that the workspace now has three layers of context working together. The copilot-instructions.md is the baseline that's always there. Skills are invoked when you need deep domain expertise on a specific algorithm or process. Prompts are invoked when you have a task to complete. Each layer does one thing rather than trying to do all three.

The skills also carry bundled assets alongside them. cfg-template.js and svg-export-template.js live in a separate templates/ folder at the workspace root — actual code files to copy from, not described templates. The model reads the file, not a description of what the file contains.

The agent — giving Copilot a persona for this work

After building the instruction file, the prompts, and the skills, there was still one thing missing. All of those layers provide context and task templates, but they don't define how Copilot should show up in a conversation. Should it ask questions or jump straight to code? Should it share aesthetic opinions or stay neutral? Should it think out loud about tradeoffs or just execute?

This is what a custom agent file does. GitHub Copilot supports .agent.md files — a way to define a specific mode of engagement for a workspace. Think of it less as a configuration file and more as a character brief. You're describing a collaborator, not setting parameters.

One of the agents created for this workspace is called Artist. A few things I wanted it to be that a generic assistant isn't:

Aesthetically opinionated. Generic Copilot will generate whatever you ask for and tell you it looks great. That's not useful in a creative context. The Plotter Artist agent is instructed to say when a composition feels unbalanced, when a result won't work well on paper, when something is genuinely beautiful. It has a point of view.

Plotter-first in its thinking. Every line in every sketch eventually becomes a physical pen move. The agent is primed to think through the physical consequences of every code decision — stroke weight for ink bleed, density for print time, no fills because the plotter can't render them. This should surface as a natural part of every code conversation rather than something to remember to check afterward.

Curious about accidents. The most interesting things that happened in this project came from something the algorithm did unexpectedly — a natural void, a lucky density distribution. The agent is instructed to ask "can we make this deliberate?" when something unexpected and good happens. That's the core creative loop in generative art: notice, name, encode.

Connected to the skills. The agent knows to load the relevant skill before making parameter recommendations — gray-scott-plotter before touching reaction-diffusion parameters, p5js-algorithm-library before discussing algorithm choices. The skills aren't useful if they're not invoked.

Responsible for the documentation. The agent is explicitly told that notable moments in a session should end up in documentation_draft.md. That's the meta layer: the work documents itself as it happens, rather than being reconstructed after the fact. I went through many iterations of testing and improvements and I wanted to gether interesting moments for the future reference and for this article.

The structure of the agent file is different from the skills and prompts in an important way. Skills are reference material — you invoke them to look something up. Prompts are task templates — you invoke them to do something. The agent file defines a mode of being that persists through an entire session. It's the difference between a tool and a collaborator.

The Artist doesn't work alone, though. There's a second agent — the Writer — whose only job is documenting the materials. The two agents form a complete loop: Artist builds and experiments, Writer documents and reflects. Neither crosses into the other's territory. That separation is what makes the whole thing work as a system rather than just a collection of instructions.

The Writer agent knows nothing about p5.js or Gray-Scott parameters. It knows the documentation: the current section order, the voice it's written in, when to add a new ## section versus a ### subsection, and what belongs in "What's next" as a placeholder. Its tools are scoped to read, edit, and search — no code execution, no web lookups. It doesn't need them.

The Artist now ends notable moments with a session brief — a small structured document with five fields: what was built, what was surprising, what was learned, which prompt or skill was involved, and where in the document this belongs. The Writer reads the brief, reads the full document, and writes prose. The Artist doesn't touch documentation_draft.md directly.

The brief format matters because it forces a moment of reflection before delegating. Writing "what was surprising: nothing" is itself an observation. Writing the suggested section placement makes you decide whether something deserves a new section or just a paragraph appended to an existing one. The handoff is lightweight but it's not zero — and that's intentional.

When Copilot chooses

Every session so far had started the same way: I brought an idea. A mood phrase, an image reference, an algorithm name. This time I didn't bring anything. The request was entirely open: choose something you'd find interesting to explore, no direction from me.

The algorithm it chose was differential growth — on the candidate list in the algorithm library but never implemented. It wasn't a random suggestion. Given access to the whole library, it found the gap and proposed to fill it.

The result was fine. Four closed curves that grow and fold under competing spring and repulsion forces, cross-curve interaction, 720 iterations, a pale botanical green palette on near-black. The mechanics were interesting — the cross-curve repulsion meant the forms negotiate shared space rather than growing independently. All workspace conventions followed correctly.

Here is how one curve looked like:

With some work this could look quite nice, but differential growth is not a simplest thing to play with, so I will keep it for later.

So I wasn't genuinely excited about the result. The algorithm is beautiful in principle and the code was clean and correct. It just didn't land with the visual impact I'd felt with the other sketches. Giving the assistant total creative freedom worked technically — it made a valid, plotter-ready sketch — but that's not the same as making something I'd want to print. This part of the setup still needs work.

When I give a direction

The difference when I brought my own starting point was immediate and obvious. The same assistant, the same prompts, the same instruction file — but starting from a feeling or an image rather than an open brief. A few sessions in a row I gave a prompt and the result was exactly what I was aiming for. Pictures to follow for each.

Water ripple — wave interference

I described it as bleaks like water ripples — a slightly wrong word that captured something real: those bright unstable flashes when sunlight hits uneven water. I didn't name an algorithm. I said what it should feel like. The Artist picked wave interference, laid out the plan, and generated the sketch. I confirmed the wave count and iso-level count. That was essentially it. The first-pass result was, in my own words at the time: exactly what I imagined. I tried to generate water ripple multiple times before, and it was never even close to this result.

Terrain mesh — perspective wireframe

This one started with a reference image — a glowing electric-blue 3D wireframe wave terrain. I shared the reference and the Artist read it directly: perspective projection, row compression toward the horizon, stroke weight graduated front to back. One parameter tweak I asked for afterward — single-octave noise instead of three, to keep the ridge lines clean and unbroken. Everything else came out right the first time. This one is nothing special from the first look, but I tried it our of curiosuty. I also tried to build terrain before but it was way too time consuming to get what I wanted, but this time, with the setup I have, it was basically done right from the first attempt.

Wire mesh — caustic water

I shared a photo of a sunlit pool mid-session and the word that came back was caustic — not as aesthetic description but as an optical phenomenon. The Artist connected it to the algorithm: when displaceScale substantially exceeds gridStep, wave crests bunch neighbouring rows into tight clusters, exactly replicating how real caustics form. Wide frequency range for chaotic multi-scale interference, eight waves. The sketch arrived essentially complete. A reference image changed not just what I was aiming for but what the algorithm was doing — and that reframing came from the agent, not from me.

What I actually built, and what surprised me

The setup I ended up with is: one instruction file, six reusable prompts, two skills, two agents. On paper that sounds like a lot of scaffolding. In practice it's nearly invisible — it just sits there and the sessions feel different.

What changed most was the quality of the starting point. Before building this, every session with Copilot began with ten minutes of re-establishing context: what p5.js mode I use, what the SVG export format looks like, what a CFG block is. With the instruction file loaded, that part is gone. The first message in a session is already a collaboration rather than an orientation.

But the thing that genuinely surprised me was how much smarter the responses got in the right context. Not smarter in a vague sense — specifically smarter. The Artist agent proposes before it generates, which means it catches mismatches before they become 300 lines of code to undo. It asks what an algorithm should feel like, not just what it should do. It loads a skill and follows a parameter reference table rather than guessing at values. That's not a different model. That's the same model with the right scaffolding around it.

The sketches in the second half of this article — the wave interference, the terrain mesh, the caustic wire — each took one or two rounds of adjustment, not ten. The agent understood what I was after from the first message because the instructions and prompts gave it enough context to make a real choice rather than a safe one. That gap between "technically correct and generic" and "actually what I wanted" is the whole gap that good context closes.

Generative art is already a process of building structure and then letting it surprise you. What this project added is another layer of the same thing: build the right instructions, prompts, and agent definitions, and then let the assistant surprise you too. The best sessions felt less like prompting a tool and more like working alongside someone who already understood the aesthetic and just needed to be shown the direction.

That's exactly what I was trying to build. I think it worked.