jai_spider: a Web Application Host​

jai_spider serves a JaiScript program as a web application. It is a host like any other on these pages: an HTTP/1.1 server, SQLite bound as a class, and a script library, in tools/jai_spider.

• The app is a program that ends in spider::spin(routes). Each request goes to the handler it spun.
• Saved edits apply to the next request. The host reaches the app's imports before every request; an edit that fails its check is logged and the app keeps serving the code it had.
• One thread runs the network loop and the script. Requests reach script one at a time, so a slow handler delays the ones behind it.
• SQL text is always a program constant; values only bind as parameters.
• It speaks no TLS. Deploy behind a reverse proxy that terminates HTTPS.

Running an app​

jai_spider tools/jai_spider/examples/guestbook/app.jai
/\(o.o)/\  jai_spider 0.5.2-beta
jai_spider: spinning tools/jai_spider/examples/guestbook/app.jai on http://127.0.0.1:8080/ (files from .../guestbook/public)
GET / 200 1.21 ms
GET /style.css 200 0.31 ms (file)
POST /sign 303 2.40 ms
jai_spider: reloaded views.jai
GET / 200 1.87 ms

The jai_spider CMake target builds bin/jai_spider beside the runner (JAISCRIPT_BUILD_SPIDER, on in a standalone build). SQLite is the system's (3.37 or newer) when CMake finds one, otherwise the official amalgamation, downloaded at configure time and checked against a pinned hash; -DJAI_SPIDER_SQLITE_ZIP=<zip> or -DJAI_SPIDER_SQLITE_DIR=<dir> supply it offline. Ctrl+C stops the server and closes every database. In the request log a control or non-ASCII byte of the path is written as \xNN.

The usual options are --port N (default 8080), --root DIR for the static root and --dev for error pages that show the error. Every option: command-line options.

A first app​

import "spider.jai";

Database db = open_database("notes.db");        // relative to the app's folder
db.execute("CREATE TABLE IF NOT EXISTS notes (id INTEGER PRIMARY KEY, text TEXT NOT NULL)");

int requests;                                   // no initializer: the count survives every reload

spider::Router routes = spider::Router();
routes.get("/", (spider::Request req) => {
    requests += 1;
    return spider::html(`<h1>${requests} requests</h1>`);
});
routes.get("/notes", (spider::Request req) => db.query("SELECT id, text FROM notes"));   // a value answers JSON
routes.post("/notes", (spider::Request req) => {
    db.execute("INSERT INTO notes (text) VALUES (?)", req.form()["text"]);
    return spider::redirect("/notes");
});
routes.get("/notes/:id", (spider::Request req) => {
    var note = db.query_one("SELECT id, text FROM notes WHERE id = ?", req.param("id"));
    return note == null ? spider::not_found() : spider::json(note);
});

spider::spin(routes);

spider::spin(handler[, options]) (also named spider::serve) takes a spider::Router or any function of a spider::Request. The options are budget (seconds a request may run) and max_body (bytes), which override the command line from the next request on; any other key raises.

Besides the library in lib/, the host registers the full standard library, the runner's file functions, the web:: library, and parse_cookies, parse_multipart, http_date and content_type_for. A page is usually a template, whose splices escape every value; a handler may return the web::html it produces.

Request and Response​

spider::Request is a script class in lib/spider.jai; its source is the reference.

Every member, uploads included: spider::Request.

spider::Response (its members) is the class a handler answers with and a client call returns. lib/spider.jai adds the helpers that build the usual answers:

spider::html(text), spider::plain_text(text) and spider::json(value) answer a body with its Content-Type, spider::redirect(url) a 303, spider::not_found(text) a 404 and spider::file(path, content_type) a file from disk. Each one's arguments and defaults: response helpers.

Routing​

spider::Router has get, post, put, patch, delete, any and add(method, pattern, handler). Routes match in the order they were added.

SQLite, injection-proof by construction​

open_database(path) opens or creates a database file (relative to the app's folder; ":memory:" for a scratch one) and returns a Database. Copies share its connection, which closes with the last of them. Files open in WAL mode.

execute, query, query_one and value take the SQL text first and the values after it, and the text is a constant argument. Text built from data does not load (argument 1 of 'query' is not a constant: SQL text must be a constant; pass values as parameters (?)), and text that reaches the call some other way is refused at run time with SQL text must be a constant; pass values as parameters (?) before SQLite sees it. There is no escape or quote function.

const string FIND_POST = "SELECT * FROM posts WHERE id = ?";
var post = db.query_one(FIND_POST, req.param("id"));
int changed = db.execute("UPDATE posts SET title = ? WHERE id = ?", title, id);
array<map<string, var>> rows = db.query("SELECT id, title FROM posts WHERE author = ?", author);
var by_name = db.query_one("SELECT * FROM posts WHERE id = :id", {"id": id});   // one map binds by name
int count = db.value("SELECT COUNT(*) FROM posts");
var picked = db.query("SELECT * FROM posts WHERE id IN (?)", [3, 5, 8]);     // an array fills IN (?)
var sorted = db.query("SELECT * FROM posts ORDER BY {} LIMIT 20",           // {} takes one constant choice
                      sql_choose(req.query.get("sort", "created"), ["created", "title", "created DESC"]));
db.transaction(() => {
    db.execute("INSERT INTO posts (title) VALUES (?)", title);
    db.execute("INSERT INTO audit (what) VALUES (?)", "post");
});

• Types. Values bind from int, float, bool, string and null. Columns come back as int, float, string (TEXT, and a BLOB as its bytes) or null.
• Statements are prepared once and kept until the database closes. A constant text of several statements given to execute without values runs them all, which suits a schema.
• transaction(fn) commits when fn returns and rolls back when it raises, then raises again; inside another transaction it is a savepoint. begin, commit, rollback, last_insert_id, changes, in_transaction, is_open and close exist too.
• Errors are catchable and carry SQLite's message: SQLite error: no such table: posts.

Sessions​

lib/session.jai keeps sessions in a table of the app's database (spider_sessions). The cookie carries only a 256-bit random id (crypto::random_hex(32)); the data stays on the server as JSON.

import "session.jai";

spider::SessionStore sessions = spider::SessionStore(db);

routes.post("/sign-in", (spider::Request req) => {
    spider::Session s = sessions.load(req);      // the visitor's session, or a new empty one
    sessions.rotate(s);                          // a fresh id once the visitor signs in
    s.set("user", req.form()["name"]);
    s.set("flash", "Welcome back.");
    spider::Response r = spider::redirect("/");
    sessions.save(s, r);                         // the row and the cookie, when the session changed
    return r;
});

spider::Session has get(key, fallback), has, set, erase and take (read once, for a message shown after a redirect). spider::SessionStore has load, save, rotate, destroy and purge_expired, and the fields cookie_name, max_age and secure_cookie (set it behind an HTTPS proxy).

Reaching other servers​

jai_spider installs the spider:: client and grants the network permission. spider::catch(url) fetches and spider::sling(content, url) sends; both return a spider::Response, which a handler can return as it came:

routes.get("/weather/:city", (spider::Request req) => {
    spider::Response upstream = spider::catch("https://weather.example/v1/now?city=" + web::url_encode(req.param("city")), {"timeout": 5});
    if (!upstream.ok()) { return spider::json({"error": "the weather service answered " + to_string(upstream.status)}, 502); }
    return upstream;                              // the upstream answer, passed through as it came
});

• A call blocks the script thread, and the requests behind it, until the answer arrives or its timeout passes. Keep upstream timeouts short.
• A passed-through answer loses its X-Sendfile header and the headers that describe one connection (Connection, Keep-Alive, Proxy-Connection, TE, Trailer, Transfer-Encoding, Upgrade, Content-Length). Its cookies pass through, so clear cookies when they are not yours to set.
• Which transport carries the call, and whether it speaks HTTPS, is in The HTTP transport.

Hot reload​

Before each request the host executes import("app.jai") (Reloading from a running program). An unchanged program costs one stamp check per file. A changed file is parsed and checked with everything it imports, then runs as its own unit:

• its functions and classes replace the old ones in place, and live instances migrate (Hot Reload);
• its top-level statements run again, so the router is rebuilt and the closing spider::spin hands it to the server;
• a variable declared without an initializer (int requests;) keeps its value.

An edit that fails to parse or check is logged once, and the app serves the code it had until the file is saved again:

jai_spider: the edit was rejected; serving the code the app had:
static check failed:
views.jai:7:25: error: cannot initialize 'n' declared 'int' with 'string'
    string footer() { int n = "three"; return "..."; }
                            ^
GET / 200 0.84 ms
jai_spider: reloaded views.jai
GET / 200 1.66 ms

Static files​

For GET and HEAD, a path that names a file under the static root is served before the app sees the request; a folder serves its index.html. A file is sent with a Content-Type from its extension, a weak ETag, Last-Modified and Cache-Control: no-cache, so a browser revalidates each time (304 when unchanged).

• A file or folder whose real name starts with a dot is never served, whatever name reaches it (a Windows 8.3 short name such as ENV~1 included), nor is anything outside the root.
• A request path with a . or .. segment, or a segment that decodes to one holding /, \ or NUL, is answered 400 before the app sees it, so a *rest capture handed to spider::file never climbs out of its folder.

Errors and limits​

HTTP/1.1 and 1.0 are served with keep-alive and pipelining, bodies by Content-Length or chunked, and Expect: 100-continue. A connection holds one response at a time: the next request is read once the last response has left.

A run-time error or an uncaught throw in a handler answers 500 and is logged. The limits on bodies, request heads, Host names, timeouts and connections each answer with their own status. Every failure with its status and logged text: failures and limits.

tools/jai_spider/examples/guestbook is a complete app: templates, a form, a session, SQLite storage, a JSON API and a stylesheet.