Proschi language reference

Proschi documents describe an architecture (nodes, groups, connections) and use cases (step-by-step request flows) as plain text. The parser lives in frontend/src/dsl/ and never throws: problems come back as diagnostics with a line and column, and everything else still renders.

The parser is the definition of the language. The same code powers the web editor, the proschi command line and the language server, so every editor reports the same problems. See Editor support for VS Code, IntelliJ, Neovim, Helix, Sublime Text and CI, and the grammar below for a summary of the syntax.

title "E-Commerce Platform"

group vpc "AWS VPC" {
  gateway "API Gateway"   [AWS API Gateway] @Platform
  orders  "Order Service" [REST API]        @Orders "Handles order processing"
}
ordersDb "Orders DB"   [PostgreSQL]
events   "OrderEvents" [Kafka]

gateway -> orders   : HTTP
orders  -> ordersDb : SQL
orders  -> events   : Publish

usecase "Create order" {
  gateway -> orders : POST /api/orders json {
    "sku": "A1"
  }
  alt "Created" {
    par {
      orders -> ordersDb : INSERT order
      orders ->> events  : OrderCreated {"orderId": "o-1"}
    }
    orders --> gateway : 201 {"status": "pending"}
  } alt "Invalid payload" {
    orders --> gateway : 400 {"error": "sku required"}
  } alt "DB down" {
    orders -x ordersDb : INSERT order
    orders --> gateway : 503
  }
}

The architecture is written once. Any number of use cases can play over it, and each use case can branch into scenarios (success and error paths) with alt. A document can also carry a high-level design: traffic, requirements, capacity, the data model, decisions and tests.

The canonical style is what proschi fmt (or Format code in the editor) produces: two spaces per block and aligned columns in runs of similar lines; see Formatting.

Statements #

• Ids are letters, digits and _, starting with a letter or _.
• Tech names a technology from the catalog (about 210 of them, see Tech stacks), e.g. [AWS Lambda], [Redis] or [Spring Boot]. It decides the node's icon, its kind and its default numbers in the simulation.
• Annotation tech stacks (Text Note, Sticky Note, Comment) make text nodes. The description, or else the name, becomes the text.

Tech stacks #

The catalog lives in frontend/src/catalog/componentCatalog.ts; the editor's and the language server's completion list all of it. Each entry has the names people usually write as aliases, so [S3], [Amazon S3] and [AWS S3] are the same node, and so are [Postgres] and [PostgreSQL], [ALB] and [AWS Load Balancer], [k8s] and [Kubernetes]. Matching ignores case, spaces and punctuation ([route 53], [pubsub], [nodejs]) and a trailing version ([PostgreSQL 16], [Redis 7.2]). The parse output always holds the catalog name.

A tech the catalog does not know is a warning, not an error, so a document that uses one still renders and simulates. The node keeps the text you wrote and is drawn and simulated as the kind its name suggests: [TigerBeetle DB] is a database, [Acme Queue] a queue, [In-house Search Index] a search node. Without a telling word it takes the kind of the closest catalog tech ([Postgress] is a database), and otherwise it is a service. It is never a client: an unknown node has a finite capacity, a latency and a price. The warning names the closest catalog tech when there is one, and the language server offers to replace the name with it:

Unknown tech stack 'Postgress'. Did you mean 'PostgreSQL'? Until then it is simulated as a generic database, like [Database]

To keep a product name the catalog lacks without the warning, use the generic tech of its kind instead ([Database], [Message Queue]) and put the product in the node's name or description.

Use case steps #

How a step label is read:

• A request label may start with x<N> (fan-out: the step happens N times per request) and ~<size> (payload size), in either order, e.g. worker -> feeds : x200 LPUSH feed:{follower} or client -> blobs : ~2MB PUT /files/{id}. They are removed before the rest of the label is read. See Fan-out and payload size.
• POST /orders sets the HTTP method and endpoint. Only the standard verbs count: GET, POST, PUT, PATCH, DELETE, HEAD, OPTIONS.
• The path may be a template with {param} segments, e.g. GET /orders/{id}. Braces inside the path are part of it, not a payload.
• After the method and path, json …, xml … or text … sets the payload format and body. A body that starts with {, [ or < is recognised without a keyword.
• A JSON payload can span several lines. It continues until its brackets balance.
• Any other text becomes the step name, e.g. INSERT order or OrderCreated {…}.
• On a response, a leading three-digit number is the status code, e.g. 201 {"id": 1}.

proschi check can compare these steps with the OpenAPI specs of the services they call: endpoints, status codes and JSON payloads. See Checking against OpenAPI.

Fan-out and payload size #

A prefix counts only at the start of a request label and only when followed by whitespace or the end of the label, so xml payload, x-request-id and GET /x200 are ordinary labels. x0, a prefix given twice and a malformed size (~2mb, ~2, ~MB) are errors at the prefix. Response (-->) labels and connection labels outside use cases are read as written.

Reads and writes #

Every request step has an access, read or write:

1. With an HTTP method: POST, PUT, PATCH and DELETE write; GET, HEAD and OPTIONS read.
2. Otherwise the first word of the label (after any prefixes, compared case-insensitively, the whole word) decides. The words in the table below write.
3. Everything else reads: GET, SELECT, QUERY, SCAN, FETCH, LOOKUP, GEOSEARCH, GetItem, MGET, an event name such as OrderPlaced, …

Words that only start like a verb (Stored, Booking, Notification) read.

durable requirements and writes X before responding count write steps only, and reads and writes load a store's read and write capacity separately.

How the protocol is inferred:

Steps should follow the architecture: when a step goes between two nodes that no connection joins (in either direction), the parser warns No connection between 'a' and 'b' in the architecture; add 'a -> b'. For a response the suggested connection points the way the request went. Calls of a node to itself are fine, and a document without any connection (only use cases) is not checked. The language server offers a quick fix that adds the connection.

Scenarios #

One endpoint rarely has one outcome. alt blocks split a use case into scenarios without copying the steps they share:

usecase "Get order" {
  gateway -> orders : GET /api/orders/42      # shared by every scenario

  alt "Found" {
    orders -> ordersDb  : SELECT order 42
    orders --> gateway : 200 {"id": 42}
  } alt "Not found" {
    orders -> ordersDb  : SELECT order 42
    orders --> gateway : 404 {"error": "not_found"}
  } alt "DB timeout" {
    orders -x ordersDb  : SELECT order 42
    orders --> gateway : 504
  }

  gateway ->> audit : OrderViewed               # shared again, after every branch
}

• alt blocks that follow each other directly are one set of alternatives. Close one and open the next on the same line (} alt "B" {) or on the next line; a step between two blocks starts a new set.
• Each branch becomes a scenario. Steps before a set are shared by all its branches, and steps after it are added to every branch.
• alt blocks can be nested. Nested sets, and several sets one after another, multiply: two sets of two branches give four scenarios, named like A › X. A use case keeps at most 32 scenarios; the parser warns past that.
• A response inside a branch can answer a request made before the branch.
• A scenario is an error path when the use case's first request is answered with a 4xx or 5xx status, or fails with -x. Playback draws error replies and failed calls in red.
• A use case without alt has a single scenario.

A branch can say when it happens: alt "Not found" when "no order has that id" { (also } alt "…" when "…" {). The condition is free text in quotes; when is a keyword only in this position, so it still works as an id elsewhere. The editor shows the condition on the scenario tab's tooltip and as a When: line during playback. For nested branches the conditions on the path are joined with ·, e.g. cache miss · db down.

Scenario ids are the slugged branch names (not-found, a-x), used in links.

Grouping by endpoint #

The use case picker in the editor groups use cases by the HTTP method and path of their first step, e.g. all POST /api/orders flows together. Use cases that don't start with an HTTP call are listed under Other flows.

Path templates. Calls to the same endpoint with different ids share a group:

1. A path with a {param} segment groups by itself: GET /orders/{id}.
2. Otherwise, a concrete path joins a template endpoint of another use case in the document with the same method that matches it, where {param} stands for exactly one non-empty segment: GET /orders/order-789 joins GET /orders/{id}.
3. Otherwise, segments that are all digits or a UUID are read as {id}: GET /orders/42 and GET /orders/43 group as GET /orders/{id}.

The use case's endpoint stays the literal path; the group key is endpointGroup in the parse output.

Imports #

Write the infrastructure once and keep use cases in other files, e.g. one per team or domain:

# infra.proschi
title "Shop infrastructure"
gateway "API Gateway" [AWS API Gateway]
orders "Order Service" [REST API] @Orders
ordersDb [PostgreSQL]
gateway -> orders
orders -> ordersDb

# checkout.proschi
title "Checkout"
import "infra.proschi"

usecase "Place order" {
  gateway -> orders : POST /orders
  orders -> ordersDb : INSERT order
  orders --> gateway : 201
}

• The path is relative to the importing file. Imports may be nested: an imported file can import others.
• Everything an imported file declares (nodes, groups, connections, use cases) becomes part of the diagram. The importing document's title names it; titles of imported files are ignored.
• A file imported along several paths (a and b both import infra) is included once. An import cycle is an error on the import that closes it: Import cycle: a.proschi → b.proschi → a.proschi. A file that does not exist is an error: Cannot find 'x.proschi'.
• Ids are shared by all files: declaring the same id in two files is an error at the later declaration, naming the file and line of the first. Use case ids stay unique too (a repeated name gets a -2 suffix).
• Undeclared ids become plain nodes only after every file is read, so a use case file may use nodes its imports declare.
• Problems in an imported file are reported with that file's path (file in proschi parse output).

Where imports are resolved:

In the web editor, nodes and connections from imported files are drawn like any other, but canvas edits only change the open document: moving, renaming or deleting something declared in an imported file is refused with a short message. Edit that file instead. The problems panel lists problems in imported files with the file name in front; clicking one opens that diagram.

High-level design #

Beyond what talks to what, a document can say how much traffic flows, how fast and reliable the system must be, what data lives where and why it is built this way. These statements are blocks at the top level; each line inside a block follows that block's own rules. The design they describe is in docs/design/hld-and-practice.md; how the simulation turns them into numbers, and how far to trust those numbers, is on How the simulation works.

title "URL Shortener" "Turns long URLs into short codes and redirects visitors"

api   "Shortener API" [REST API]   x12
cache "Code cache"    [Redis]      x2
db    "URL store"     [PostgreSQL] x3
# … connections and the use cases "Redirect" and "Shorten"

traffic {
  "Redirect" 100k rps mix "Cache hit" 90%, "Cache miss" 10%
  "Shorten"  1k rps
}

requirements {
  p99 "Redirect" < 100ms
  availability >= 99.9%
  durable "Shorten"
  survive any node failure
  cost <= 3000 usd/month
}

capacity {
  db 20k rps latency 4ms
}

entity Url in db "One short code and where it points" {
  code      string key
  target    string
  createdAt time   index
}

decision "Cache redirects in Redis" {
  because "Reads outnumber writes 100:1 and p99 must stay under 100 ms"
  rejected "Read replicas only" "About 5 ms per read and many replicas at 100k rps"
}

test "Redirects are served from the cache" {
  "Redirect" calls any cache before any database
  "Redirect" scenario "Cache hit" never calls any database
}

The URL shortener HLD example in the editor is a complete document using every statement.

The words that open these blocks are keywords only in that shape (traffic {, entity Name …, decision "…" { or decision "…" because, test "…" {), so older documents that use them as node ids, e.g. test "Test runner" [REST API], still parse. A malformed line inside a block is an error on that line and the rest of the block is still read; a misplaced block (inside a group or a use case) is an error and its lines are skipped. Imported files contribute all of these sections. In the parse output each section is left out when the document has none.

Quantities #

A quantity is a number, an optional magnitude and an optional unit:

The unit may be attached (50ms, 99.9%, 100krps) or one space away (50 ms, 100k rps). Rates are normalised to requests per second (6k rpm is 100 rps), durations to milliseconds (1.5s is 1500 ms) and bandwidth to megabytes per second (1 GB/s is 1000 MB/s). ms is always milliseconds. A statement that wants a unit reports a quantity without one, or with the wrong one: '100k' needs a unit: expected a rate, e.g. 100k rps, 6k rpm or 1m rpd.

Traffic #

traffic {
  "Redirect" 100k rps mix "Cache hit" 90%, "Cache miss" 10%
  "Shorten"  1k rps
}

• One line per use case: "<use case>" <rate>.
• mix splits the use case's traffic over its scenarios by their full names (A › B for nested branches; a use case without alt has one scenario, named like the use case). Without mix, all traffic goes to the first scenario.
• Shares should add up to 100%. Otherwise there is a warning and they are scaled to fit. In the parse output shares are fractions (0.9).
• Unknown use case or scenario names are warnings, checked once every file is read. A second line for the same use case is an error.

Requirements #

requirements {
  p99 "Redirect" < 50ms
  p99 "Redirect" scenario "Cache hit" < 10ms
  p95 < 300ms                 # every use case
  availability "Redirect" >= 99.95%
  availability >= 99.9%       # every use case
  durable "Shorten"
  survive any node failure
  survive failure of cache    # a node id, [Tech], or any <kind>
  cost <= 3000 usd/month
}

<= is also accepted for latency and > for availability. Each requirement becomes a test that the simulation evaluates.

Capacity #

capacity {
  db    20k rps latency 4ms availability 99.95% cost 400 usd/month durable
  cache 150k rps
  users reads 30k rps writes 8k rps shards 4 consistency strong
  blobs bandwidth 500 MB/s egress 0.05 usd/GB
  pay   timeout 300ms
}

Per-replica overrides of the default profile of a node, one line per node id, parts in any order:

An unknown node id is a warning, and so are shards or consistency on a node that is not a data store. A part given twice, a rate together with reads or writes, and a second line for the same node are errors.

Entities #

entity Url in db "One short code and where it points" {
  code      string key
  target    string
  createdAt time   index
}

• entity <Name> [in <node id>] ["Description"] {, then one field per line: <name> <type> {flag} with the flags key, index, unique and optional. Types are free identifiers: string, int, time, uuid, json, …
• in <node> places the entity in a store. It is a warning when that node is not a data store: a cache, database, search, analytics, queue or storage node (see kinds).

Decisions #

decision "Cache redirects in Redis" {
  because "Reads outnumber writes 100:1 and p99 must stay under 50 ms"
  rejected "Read replicas only" "About 5 ms per read and many replicas at 100k rps"
  rejected "Memcached" "No replication; losing a node empties the cache"
}
decision "Base62 codes from a counter" because "Short, unique, no collisions to retry"

because "<reason>" at most once, and any number of rejected "<option>" "<reason>". The one-line form takes only because.

Tests #

Tests check how the design works. Every assertion line in a test must hold.

test "Redirect is served from the cache" {
  "Redirect" calls any cache before any database
  "Redirect" scenario "Cache hit" never calls any database
}
test "Clients only enter through the gateway" {
  no path from client to any database
}
test "Checkout answers before slow work" {
  "Checkout" never waits for any queue or any external
  "Checkout" calls ledger after gateway
  any service never calls blobs
  "Retry charge" starts at any queue
}

U and S are a use case and a scenario name in quotes. An assertion that starts with a quoted use case is about that use case ("U" calls X: some step reaches X); one that starts with a selector or in "U" is about the sender (X calls Y: a step sent by X). has scenario, handles failure of and starts at are about the whole use case and take no scenario. Order in a scenario is the sequence order, requests and responses interleaved. Unknown use cases, scenarios (after scenario) and nodes are warnings; a test without assertions is a warning and two tests with one name are an error.

Selectors and kinds #

X and Y above, and survive failure of, take a selector:

A node's kind comes from its tech stack (an unknown tech: from its name, see Tech stacks):

Editing on the canvas #

The text is the source of truth. Edits on the diagram are written back into it:

A node that was only referenced, never declared, gets a declaration line above the first use case.

Links #

The address bar always holds the whole document: #code=…. When the document imports other files, their text travels along as &imports=… (a compressed map of path to source), so the link renders the same for someone who has none of the files. Those files stay attached to the diagram opened from the link and win over saved diagrams of the same name. While a use case is playing, the link also names the use case, the scenario (for use cases with alt blocks) and the step, e.g. #code=…&uc=create-order&alt=db-down&step=2, so a shared link opens playback at that step of that scenario.

Grammar #

A summary in EBNF. The language is line-oriented: each statement takes one line, except a step or connection label whose JSON payload continues until its brackets balance. Whitespace between tokens is ignored.

document     = { line } ;
line         = [ statement ] [ comment ] newline ;

statement    = title | import | node | connection
             | group-open | usecase-open | par-open | alt-open | close
             | traffic | requirements | capacity | entity | decision | test ;

title        = "title" , ( string | id ) , [ string ] ;   (* 2nd string: summary *)
import       = "import" , string ;                      (* a relative file path *)
node         = id , { string | tech | team | position | replicas } ;  (* 1st string: name, 2nd: description *)
connection   = id , arrow , id , [ ":" , label ] ;
group-open   = "group" , id , [ string ] , [ tech ] , [ position ] , "{" ;
usecase-open = "usecase" , ( string | id ) , [ string ] , "{" ;
par-open     = "par" , "{" ;
alt-open     = "alt" , ( string | id ) , [ "when" , string ] , "{" ;
close        = "}" , [ alt-open ] ;

traffic      = "traffic" , "{" , { string , quantity , [ "mix" , share , { "," , share } ] } , "}" ;
share        = string , quantity ;                       (* "Cache hit" 90% *)
requirements = "requirements" , "{" , { requirement } , "}" ;
requirement  = percentile , [ string , [ "scenario" , string ] ] , "<" , quantity
             | "availability" , [ string ] , ">=" , quantity
             | "durable" , string
             | "survive" , ( "any" , "node" , "failure" | "failure" , "of" , selector )
             | "cost" , "<=" , quantity ;
percentile   = "p50" | "p90" | "p95" | "p99" | "p999" ;
capacity     = "capacity" , "{" , { id , { capacity-part } } , "}" ;
capacity-part = quantity | "reads" , quantity | "writes" , quantity | "shards" , integer
             | "latency" , quantity | "availability" , quantity | "cost" , quantity
             | "durable" | "volatile" | "consistency" , ( "strong" | "eventual" )
             | "bandwidth" , quantity | "egress" , quantity | "timeout" , quantity ;
entity       = "entity" , id , [ "in" , id ] , [ string ] , "{" , { id , id , { flag } } , "}" ;
flag         = "key" | "index" | "unique" | "optional" ;
decision     = "decision" , string , ( "because" , string
             | "{" , { "because" , string | "rejected" , string , string } , "}" ) ;
test         = "test" , string , "{" , { assertion } , "}" ;
assertion    = string , [ "scenario" , string ] , ( "calls" , selector , [ ( "before" | "after" ) , selector ]
             | "every" , "scenario" , "calls" , selector | "never" , "calls" , selector
             | "never" , "waits" , "for" , selector
             | "writes" , selector , "before" , "responding" | "responds" , status )
             | string , "has" , "scenario" , string
             | string , "handles" , "failure" , "of" , selector
             | string , "starts" , "at" , selector
             | [ "in" , string ] , selector , [ "never" ] , "calls" , selector
             | "no" , "path" , "from" , selector , "to" , selector
             | selector , "has" , "replicas" , ">=" , integer ;
selector     = selector-atom , { "or" , selector-atom } ;
selector-atom = id | tech | "any" , kind | "any" , ( "strong" | "eventual" ) , "store" ;
kind         = "client" | "edge" | "service" | "function" | "cache" | "database"
             | "search" | "analytics" | "queue" | "storage" | "external" | "other" ;
status       = digit , digit , digit | digit , "xx" ;     (* 201, 4xx *)

arrow        = "->" | "->>" | "-->" | "-x" ;
position     = "pos" , integer , "," , integer ;
replicas     = "x" , digit , { digit } ;               (* one token, e.g. x3 *)
quantity     = number , [ magnitude ] , [ unit ] ;     (* 50ms, 100k rps, 99.9% *)
number       = digit , { digit } , [ "." , digit , { digit } ] ;
magnitude    = "k" | "m" | "b" ;
unit         = "rps" | "rpm" | "rpd" | "ms" | "s" | "%" | "usd/month" | "MB/s" | "GB/s" | "usd/GB" ;
id           = ( letter | "_" ) , { letter | digit | "_" } ;
string       = '"' , { character - '"' | "\" , character } , '"' ;
tech         = "[" , { character - "]" } , "]" ;
team         = "@" , { letter | digit | "_" | "-" } ;
integer      = [ "-" ] , digit , { digit } ;
comment      = "#" , { character } ;            (* only where a token can start *)
label        = [ label-prefix , [ label-prefix ] ] , { character } ;  (* to end of line; see below *)
label-prefix = ( "x" , digit , { digit } | "~" , number , size-unit ) , whitespace ;  (* requests only *)
size-unit    = "B" | "KB" | "MB" | "GB" | "TB" ;

Where each statement may appear:

-x is only valid as a step. A label is read as described in Use case steps: optional x<N> / ~<size> prefixes on a request, an optional HTTP method and path, an optional json / xml / text payload, and on a response a leading status code.

The grammar describes syntax only. The parser also checks meaning: duplicate ids, unknown tech stacks (a warning), responses without a matching request, steps between nodes the architecture does not connect, import cycles and missing imported files, unknown use cases, scenarios and nodes in the high-level design sections, and so on. Those checks are what the language server and proschi check report.