hawk/README.md

# Hawk - Embeddable AWK Interpreter in C/C++

`Hawk` is a stable and embeddable **AWK interpreter written in C**.
It can run AWK scripts inside your own applications or as a standalone AWK engine.
The library is stable, portable, and designed for projects that need a scripting engine with a small footprint.

![Hawk](hawk.png)

## Table of Contents
- [Features](#features)
- [Building Hawk From Source Code](#building-hawk-from-source-code)
- [Embedding Hawk in C Applications](#embedding-hawk-in-c-applications)
- [Embedding Hawk in C++ Applications](#embedding-hawk-in-c-applications-1)
- [Language](#language)
	- [What Hawk Is](#what-hawk-is)
	- [Running Hawk](#running-hawk)
	- [Execution Model](#execution-model)
		- [@pragma entry](#pragma-entry)
	- [Values and Types](#values-and-types)
	- [Expressions and Operators](#expressions-and-operators)
		- [Arithmetic and Comparison](#arithmetic-and-comparison)
		- [Strings and Regex](#strings-and-regex)
		- [Logical Operators](#logical-operators)
		- [Bitwise Operators](#bitwise-operators)
	- [Variables and Scope](#variables-and-scope)
		- [Arrays and Maps](#arrays-and-maps)
		- [Functions](#functions)
			- [Function Literals](#function-literals)
		- [Control Flow](#control-flow)
			- [if / else](#if--else)
			- [while](#while)
		- [do ... while](#do--while)
		- [for](#for)
		- [for (i in array)](#for-i-in-array)
		- [in operator (key existence)](#in-operator-key-existence)
		- [switch](#switch)
		- [break / continue / return / exit / @abort](#break--continue--return--exit--abort)
		- [nextfile / nextofile](#nextfile--nextofile)
	- [Input, Output, and Pipes](#input-output-and-pipes)
	- [Built-in Variables](#built-in-variables)
	- [Built-in Functions](#built-in-functions)
	- [Pragmas](#pragmas)
		- [@pragma entry](#pragma-entry-1)
		- [@pragma implicit](#pragma-implicit)
		- [@pragma defermodsym](#pragma-defermodsym)
		- [@pragma striprecspc](#pragma-striprecspc)
	- [@include and @include\_once](#include-and-include_once)
	- [Comments](#comments)
	- [Reserved Words](#reserved-words)
	- [Some Examples](#some-examples)
	- [Garbage Collection](#garbage-collection)
	- [Modules](#modules)
		- [Hawk](#hawk)
		- [String](#string)
		- [System](#system)
		- [ffi](#ffi)
		- [mysql](#mysql)
		- [sqlite](#sqlite)
	- [Incompatibility with AWK](#incompatibility-with-awk)
		- [Parameter passing](#parameter-passing)
		- [Positional variable expression](#positional-variable-expression)
		- [Return value of getline](#return-value-of-getline)
		- [Others](#others)

## Features

- Full AWK interpreter - mostly POSIX AWK compatible, with additional extensions.
- Embeddable library - integrate AWK scripting into C or C++ projects as an execution engine.
- C and C++ APIs - core functions exposed in C, with convenient C++ wrapper classes available.
- Flexible usage - usable as both a standalone command-line interpreter and a library.
- Portable core - the base library depends only on the standard C library.
- Optional extensions - loadable modules (e.g. MySQL access, FFI) can be built in or used via shared objects.
- Mature and stable - developed and maintained for many years with proven reliability.
- Embedded sed functionality - includes a sed engine that can be used from C/C++ or invoked via the CLI using --sed <options>

# Building Hawk From Source Code

Hawk uses `autoconf` and `automake` for building. Run the following commands to configure and compile Hawk:

```sh
$ ./configure ## This step offers various build options
$ make
$ make install
```

# Embedding Hawk in C Applications

Here's an example of how Hawk can be embedded within a C application:

```c
#include <hawk.h>
#include <stdio.h>
#include <string.h>

static const hawk_bch_t* src =
	"BEGIN { print ARGV[0];"
	"   for (i=2;i<=9;i++)"
	"   {"
	"       for (j=1;j<=9;j++)"
	"           print i \"*\" j \"=\" i * j;"
	"       print \"---------------------\";"
	"   }"
	"}";

int main ()
{
	hawk_t* hawk = HAWK_NULL;
	hawk_rtx_t* rtx = HAWK_NULL;
	hawk_val_t* retv;
	hawk_parsestd_t psin[2];
	int ret;

	hawk = hawk_openstd(0, HAWK_NULL); /* create a hawk instance */
	if (!hawk)
	{
		fprintf(stderr, "ERROR: cannot open hawk\n");
		ret = -1; goto oops;
	}

	/* set up source script file to read in */
	memset(&psin, 0, HAWK_SIZEOF(psin));
	psin[0].type = HAWK_PARSESTD_BCS;  /* specify the first script path */
	psin[0].u.bcs.ptr = (hawk_bch_t*)src;
	psin[0].u.bcs.len = hawk_count_bcstr(src);
	psin[1].type = HAWK_PARSESTD_NULL; /* indicate the no more script to read */

	ret = hawk_parsestd(hawk, psin, HAWK_NULL); /* parse the script */
	if (ret <= -1)
	{
		hawk_logbfmt(hawk, HAWK_LOG_STDERR, "ERROR(parse): %js\n", hawk_geterrmsg(hawk));
		ret = -1; goto oops;
	}

	/* create a runtime context needed for execution */
	rtx = hawk_rtx_openstd(
		hawk,
		0,
		HAWK_T("hawk02"), /* ARGV[0] */
		HAWK_NULL,  /* stdin */
		HAWK_NULL,  /* stdout */
		HAWK_NULL   /* default cmgr */
	);
	if (!rtx)
	{
		hawk_logbfmt(hawk, HAWK_LOG_STDERR, "ERROR(rtx_open): %js\n", hawk_geterrmsg(hawk));
		ret = -1; goto oops;
	}

	/* execute the BEGIN/pattern-action/END blocks */
	retv = hawk_rtx_loop(rtx); /* alternatively, hawk_rtx_exec(rtx, HAWK_NULL, 0) */
	if (!retv)
	{
		hawk_logbfmt(hawk, HAWK_LOG_STDERR, "ERROR(rtx_loop): %js\n", hawk_geterrmsg(hawk));
		ret = -1; goto oops;
	}

	/* lowered the reference count of the returned value */
	hawk_rtx_refdownval(rtx, retv);
	ret = 0;

oops:
	if (rtx) hawk_rtx_close(rtx); /* destroy the runtime context */
	if (hawk) hawk_close(hawk); /* destroy the hawk instance */
	return -1;
}
```

Embedding Hawk within an application involves a few key steps:

- Creating a Hawk Instance: The `hawk_openstd()` function is used to create a new instance of the Hawk interpreter, which serves as the entry point for interacting with Hawk from within the application.
- Parsing Scripts: The application can provide Hawk scripts as string literals or read them from files using the `hawk_parsestd()` function. This associates the scripts with the Hawk instance for execution.
- Creating a Runtime Context: A runtime context is created using `hawk_rtx_openstd()`, encapsulating the state and configuration required for script execution, such as input/output streams and other settings.
- Executing the Script: The `hawk_rtx_loop()` or `hawk_rtx_exec()` functions are used to execute the Hawk script within the created runtime context, returning a value representing the result of the execution.
- Handling the Result: The application can check the returned value for successful execution and handle any errors or results as needed.
- Cleaning Up: Finally, the application cleans up by closing the runtime context and destroying the Hawk instance using `hawk_rtx_close()` and `hawk_close()`, respectively.

By following this pattern, applications can seamlessly embed the Hawk interpreter, leveraging its scripting capabilities and data manipulation functionality while benefiting from its portability, efficiency, and extensibility.

Assuming the above sample code is stored in `hawk02.c` and the built Hawk library has been installed properly, you may compile the sample code by running the following commands:

```sh
$ gcc -Wall -O2 -o hawk02 hawk02.c -lhawk
```

The actual command may vary depending on the compiler used and the `configure` options used.

## Embedding Signal Handling

Signal handling is provided via a callback hook. The core library does not install OS-level signal handlers; the embedding application must do that and notify the runtime.

- Register a runtime callback with `hawk_rtx_pushecb()` and fill `hawk_rtx_ecb_t.sigset` (type `hawk_rtx_ecb_sigset_t`).
- Your callback is invoked when `sys::signal()` sets or clears a handler; use it to install or reset the OS signal handler.
- When the OS handler runs, call `hawk_rtx_raisesig()` on the target runtime.

Example:

```c
static hawk_rtx_t* g_rtx = HAWK_NULL;

static void on_os_signal (int sig)
{
	if (g_rtx) hawk_rtx_raisesig(g_rtx, sig);
}

static void on_sigset (hawk_rtx_t* rtx, int sig, hawk_fun_t* fun)
{
	if (fun)
	{
		g_rtx = rtx;
		signal(sig, on_os_signal);
	}
	else
	{
		if (g_rtx == rtx) g_rtx = HAWK_NULL;
		signal(sig, SIG_DFL);
	}
}

hawk_rtx_ecb_t ecb;
memset(&ecb, 0, HAWK_SIZEOF(ecb));
ecb.sigset = on_sigset;
hawk_rtx_pushecb(rtx, &ecb);
```

# Embedding Hawk in C++ Applications

Hawk can also be embedded in C++ applications. Here's an example:

```c++
#include <Hawk.hpp>
#include <stdio.h>

int main ()
{
	HAWK::HawkStd hawk;

	if (hawk.open() <= -1)
	{
		fprintf(stderr, "unable to open hawk - %s\n", hawk.getErrorMessageB());
		return -1;
	}

	HAWK::HawkStd::SourceString s("BEGIN { print \"hello, world\"; }");
	if (hawk.parse(s, HAWK::HawkStd::Source::NONE) == HAWK_NULL)
	{
		fprintf(stderr, "unable to parse - %s\n", hawk.getErrorMessageB());
		hawk.close();
		return -1;
	}

	HAWK::Hawk::Value vr;
	hawk.loop(&vr);  // alternatively, hawk.exec(&vr, HAWK_NULL, 0);

	hawk.close();
	return 0;
}
```

Embedding Hawk within a C++ application involves the following key steps:

- Creating a Hawk Instance: Create a new instance of the Hawk interpreter using the `HAWK::HawkStd` class.
- Parsing Scripts: Provide Hawk scripts as strings using the `HAWK::HawkStd::SourceString` class, and parse them using the `hawk.parse()` method.
- Executing the Script: Use the `hawk.loop()` or `hawk.exec()` methods to execute the Hawk script, returning a value representing the result of the execution.
- Handling the Result: Handle the returned value or any errors that occurred during execution.
- Cleaning Up: Clean up by calling `hawk.close()` to destroy the Hawk instance.

The C++ classes are inferior to the C equivalents in that they don't allow creation of multiple runtime contexts over a single hawk instance.

# Language

## What Hawk Is

Hawk is an embeddable awk interpreter with extensions. It can run hawk/awk scripts from the CLI or from C/C++ and provides modules like `str::`, `sys::`, `ffi::`, `mysql::`, and `sqlite::`.


## Running Hawk

Run a script file:

```sh
$ hawk -f script.hawk input.txt
```

Run an inline program:

```sh
$ echo "a,b,c" | hawk 'BEGIN{FS=","} {print $2}'
```

## Execution Model

Hawk follows the awk pipeline:

- Input is read as records (usually lines). `RS` controls record separation.
- Each record (`$0`) is split into fields `$1`, `$2`, ... by `FS`.
- A script is a sequence of `pattern { action }` blocks.
- `BEGIN` runs before input; `END` runs after input.

Example:

```awk
BEGIN { FS=","; print "start" }
$3 ~ /ERR/ { print NR, $1, $3 }
END { print "done", NR }
```

### @pragma entry

Hawk can override the default `BEGIN`/pattern/`END` flow with a custom entry point:

```awk
@pragma entry main
function main(a, b) {
	print "entry:", a, b
}
```

Run:

```sh
$ hawk -f script.hawk one two
entry: one two
```

## Values and Types

Hawk is dynamically typed:

- Numbers: integer and floating-point.
- Strings: Unicode text.
- Characters can be written with single quotes (e.g., `'A'`) or with double quotes prefixed with `@c` (e.g. `@c"A"`).
- Byte strings: raw bytes (`@b"..."`).
- Byte characters use `@b'X'` or `@bc"X"` and must fit in a single byte.
- Containers: array, map.
- `@nil` represents null.
- `@true` represents true.
- `@false` represents false.

Examples:

```awk
BEGIN {
	a = 10
	b = 3.14
	s = "hello"
	c = 'X'
	c2 = @C"X"
	bc = @b'x'
	bc2 = @bc"x"
	bs = @b"\x00\x01"
	m = @{"k": 1, "j": "hello"}
	arr = @["x", "y"]
	print hawk::typename(a), hawk::typename(b), hawk::typename(s), hawk::typename(c), hawk::typename(c2), hawk::typename(bc), hawk::typename(bc2), hawk::typename(bs), hawk::typename(m), hawk::typename(arr)
}
```

## Expressions and Operators

### Arithmetic and Comparison

- Arithmetic: `+`, `-`, `*`, `/`, `\`(integer division), `%`(modulo), `**` or '^'(exponentiation), `++`, `--`, `<<`, `>>`.
- Compound assignment: `=`, `+=`, `-=`, `*=`, `/=`, `\=`, `%=`, `^=` or `**=`, `%%=`.
- Comparisons: `==`, `!=`, `<`, `<=`, `>`, `>=`.
- Type-precise compare: `===` and `!==`.

Example:

```awk
BEGIN {
	x = 10 + 5 * 2
	x += 3
	s = "ha"
	s %%= "wk"
	if (x >= 20) print x
	print s
	if ("10" === 10) print "no"
}
```

### Strings and Regex

- Concatenation by adjacency: `"a" "b"`.
- Explicit concatenation: `"a" %% "b"`.
- Regex match: `~` and `!~`.

Example:

```awk
BEGIN {
	print "hi" %% "!"
	if ("A" ~ /^[A-Z]$/) print "regex ok"
}
```

### Logical Operators

- Logical AND/OR: `&&`, `||`.
- Boolean results are numeric (`0` or `1`).

Example:

```awk
BEGIN {
	if (1 && 0) print "no"; else print "ok"
}
```

### Bitwise Operators

- Bitwise AND/OR/XOR: `&`, `|`, ^^
- Unary bitwise negation: ~
- Compound bitwise assignment: `&=`, `|=`, `^^=`, `<<=`, `>>=`
- `|` also denotes pipes, so use parentheses when you mean bitwise OR.
- `>>` is also used for append redirection; use parentheses when you mean right shift.

Bitwise OR vs pipe example:

```awk
BEGIN {
	print (1 | 2)  # bitwise OR => 3
	print 1 | 2    # pipe to external command "2"
}
```

## Variables and Scope

- Variables are created on assignment.
- `@local` and `@global` declare scope explicitly.
- `@const` defines a compile-time constant evaluated at parse time.

You can initialize `@global` and `@local` at declaration time. Initializers are expressions and can reference earlier constants and globals. `@const` values are fixed and cannot be reassigned.

Example:

```awk
@const C = 5
@global g = 10 + C, h
BEGIN {
	@local x = g + 1, y
	print x, g
	x = 1
	g = 2
	print x, g
}
```

## Arrays and Maps

Hawk supports arrays and maps.

- Arrays are indexed by numbers.
- Maps accept string and numeric keys.
- Constructors: `@[]`, `@{}`, `hawk::array()`, `hawk::map()`.
- All constructors accept initial values.

Example:

```awk
BEGIN {
	arr = @["a", "b", "c"]
	m = @{"k": "v", 10: "ten"}
	arr[4] = "d"
	m["x"] = 99
	print arr[1], m["k"], m[10]
}
```

## Functions

Define functions with `function name(...) { ... }`.

- Missing args are `@nil`.
- Use `&` for call-by-reference.
- Use `...` for varargs and access them via `@argc` and `@argv`.
- Functions are first-class values and can be passed as parameters (e.g., a comparator for `asort`).

Example:

```awk
function inc(&x) { x += 1 }
function greet(name) { if (name == "") name = "world"; print "hi", name }
BEGIN { n = 1; inc(n); greet(); greet("hawk"); print n }
```

Varargs example:

```awk
function dump(...) {
	@local i
	for (i = 0; i < @argc; i++) print @argv[i]
}
BEGIN { dump("a", 10, "b") }
```

Function-parameter example:

```awk
function desc(a, b) { return b - a }
BEGIN {
	@local a, b, i
	a = @[3, 1, 2]
	asort(a, b, desc)
	for (i in b) print i, b[i]
}
```

### Function Literals

Hawk supports unnamed function literals with `func(...) { ... }`.
You can assign them to `@global`, `@local`, implicit/plain variables, and indexed elements.

- Function literals do not capture outer local state (no closure).
- They behave like normal user-defined functions used as values.

Example:

```awk
@pragma xcall on

@global gf = func(a, b) { return a + b }

BEGIN {
	@local lf = func(x) { return x * 2 }
	pf = func(z) { return z - 1 }      # plain/implicit variable
	m = @{}
	m["call"] = func(v) { return v * v }  # indexed assignment
	m.call2 = func(v) { return v + 100 }  # dot-notation indexed assignment

	print gf(10, 20), lf(7), pf(9), m["call"](5), m.call2(5)
}
```


## Control Flow

Hawk supports standard awk control flow.

### if / else

```awk
{ if ($1 > 0) print $1; else print "skip" }
```

### while

```awk
BEGIN {
	i = 1
	while (i <= 3) { print i; i++ }
}
```

### do ... while

```awk
BEGIN {
	i = 0
	do { print i; i++ } while (i < 3)
}
```

### for

```awk
BEGIN {
	for (i = 1; i <= 3; i++) print i
}
```

### for (i in array)

```awk
BEGIN {
	arr = @["x", "y"]
	for (i in arr) print i, arr[i]
}
```

### in operator (key existence)

Use `x in b` to test if a key/index exists in a map or array.

```awk
BEGIN {
	b = @{"k": 1}
	if ("k" in b) print "yes"
}
```

### switch

```awk
BEGIN {
	x = 2
	switch (x) {
	case 1: print "one"; break;
	case 2: print "two"; break;
	default: print "other";
	}
}
```

### break / continue / return / exit / @abort

```awk
BEGIN {
	for (i = 1; i <= 5; i++) {
		if (i == 3) continue
		if (i == 5) break
		print i
	}
	exit 0
}
```

`exit` terminates a script, but `END` blocks are still executed if it's not called from an `END` block.

```awk
BEGIN {
	print "BEGIN"
	exit 99
}
END {
	print "END" ## this is executed
}
```

`@abort` terminates a script immediately, skipping `END` blocks if it's not called from an `END` block.

```awk
BEGIN {
	print "BEGIN"
	@abort 99
}
END {
	print "END" ## this is not executed
}
```

Note: Hawk allows `return` inside `BEGIN` and `END` blocks, in addition to functions.

### nextfile / nextofile

`nextfile` skips the rest of the current input file (standard awk behavior). `nextofile` advances to the next output file specified with `-t`.

Example:

```sh
$ hawk -t /tmp/1 -t /tmp/2 'BEGIN { print 10; nextofile; print 20 }'
```

This writes `10` to `/tmp/1` and `20` to `/tmp/2`.


## Input, Output, and Pipes

- `getline` reads records.
- `getbline` reads records as bytes.
- `getline`/`getbline` return `1` on success, `0` on EOF, and `-1` on error.
- Redirection works with `<`, `>`, and `>>`.
- Pipes: `cmd | getline var` and `print x | "cmd"`.
- Two-way pipes: `|&`
- CSV-style field splitting is supported when `FS` begins with `?` followed by four characters (separator, escaper, left quote, right quote).

Example:

```awk
BEGIN {
	while (("ls -laF" | getline x) > 0) print "\t", x;
	close ("ls -laF");
}
```

Two-way pipe example:


```awk
BEGIN {
	cmd = "sort";
	data = hawk::array("hello", "world", "two-way pipe", "testing");

	for (i = 1; i <= length(data); i++) print data[i] |& cmd;
	close(cmd, "to");

	while ((cmd |& getline line) > 0) print line;
	close(cmd);
}
```

Redirection examples:

```awk
BEGIN {
	while ((getline line < "input.txt") > 0) print line > "out.txt"
	print "more" >> "out.txt"
}
```

Byte-record example:

```awk
BEGIN { getbline b < "bin.dat"; print str::tohex(b) }
```

CSV-style `FS` example:

```awk
BEGIN { FS="?,\"\"\""; }
{ for (i = 0; i <= NF; i++) print i, "[" $i "]"; }
```

This example splits `hawk,can,read,"a ""CSV"" file",.` to 5 fields.
- hawk
- can
- read
- a "CSV" file
- .

## Built-in Variables

Common built-ins:

- `NR`, `FNR`, `NF`
- `FS`, `RS`, `OFS`, `ORS`
- `FILENAME`, `OFILENAME`
- `ARGC`, `ARGV`
- `ENVIRON`
- `PIPECLOEXEC`

Example:

```awk
{ print NR, NF, $0 }
```

Runtime-specific predefined variables:

| Name | Type | Description |
|------|------|-------------|
| `ARGC` | int | number of command-line arguments available via `ARGV` |
| `ARGV` | array | command-line argument vector. `ARGV[0]` is usually the program name |
| `ENVIRON` | map | process environment as a map. Values imported from the host environment may become `int`, `flt`, or `str` depending on content |
| `PIPECLOEXEC` | int/bool-like | controls whether file descriptors are opened with `CLOEXEC` when Hawk runs an external command for piping. `0` keeps the default inheritance behavior, `1` enables close-on-exec. This corresponds to `@pragma pipecloexec` |

`ENVIRON` affects child processes started by piping and `sys::system()`. Modifying it changes the environment passed to those children without changing the process-global environment of the embedding application.

Examples:

```awk
BEGIN {
	print ARGC, ARGV[0]
	print ENVIRON["HOME"]
	PIPECLOEXEC = 1
}
```

## Built-in Functions

Hawk includes awk built-ins (e.g., `length`, `substr`, `split`, `index`) plus extensions in modules (see below).

Example:

```awk
BEGIN { print length("hawk"), substr("hawk", 2, 2) }
```

## Pragmas

`@pragma` controls parser/runtime behavior. File-scope pragmas apply per file; global-scope pragmas appear once across all files.

| Name          | Scope  | Values        | Default | Description                                             |
|---------------|--------|---------------|---------|---------------------------------------------------------|
| entry         | global | function name |         | change the program entry point                          |
| implicit      | file   | on, off       | on      | allow undeclared variables                              |
| defermodsym   | file   | on, off       | off     | defer resolving `mod::symbol` references to runtime     |
| multilinestr  | file   | on, off       | off     | allow a multiline string literal without continuation   |
| pipecloexec   | global | on, off       | off     | set CLOEXEC when running an external command for piping |
| rwpipe        | file   | on, off       | on      | allow the two-way pipe operator `\|&`                   |
| striprecspc   | global | on, off       | off     | removes leading and trailing blank fields in splitting a record if FS is a regular expression mathcing all spaces |
| stripstrspc   | global | on, off       | on      | trim leading and trailing spaces when converting a string to a number |
| numstrdetect  | global | on, off       | off     | convert a numeric string to a value of a numeric type automatically |
| stack_limit   | global | number        | 5120    | specify the runtime stack size measured in the number of values |
| xcall         | file   | on, off       | off     | allow one-level function call after dot or bracket index access (e.g. `m.f(...)`, `m["f"](...)`) |

### @pragma entry

Sets a custom entry function instead of the default `BEGIN`/pattern/`END` flow.

```awk
@pragma entry main;
function main () { print "hello, world"; }
```

Arguments passed on the command line are provided to the entry function:

```awk
@pragma entry main
function main(arg1, arg2) {
	print "Arguments:", arg1, arg2
}
```

```sh
$ hawk -f main.hawk arg1_value arg2_value
```

If you don't know the number of arguments in advance, use `...` and `@argv`/`@argc`:

```awk
@pragma entry main
function main(...) {
	@local i
	for (i = 0; i < @argc; i++) printf("%s:", @argv[i])
	print ""
}
```

```sh
$ hawk -f main.hawk 10 20 30 40 50
```

Named arguments can be combined with `...` to require a minimum number of parameters:

```awk
function x(a, b, ...) {
	print "a=", a, "b=", b, "rest=", (@argc - 2)
}
BEGIN { x(1, 2, 3, 4) }
```

### @pragma implicit

Controls implicit variable declaration. `off` requires `@local`/`@global`.

```awk
@pragma implicit off;
BEGIN {
    a = 10; ## syntax error - undefined identifier 'a'
}
```

In the example above, the `@pragma implicit off` directive is used to turn off implicit variable declaration. As a result, attempting to use the undeclared variable a will result in a syntax error.

```awk
@pragma implicit off;
BEGIN {
    @local a;
    a = 10; ## syntax ok - 'a' is declared before use
}
```

### @pragma defermodsym

Defers module symbol resolution to runtime.
This is useful when a script can reference optional modules that may not exist in every deployment.

```awk
@pragma defermodsym on
BEGIN {
	if (0) optionalmod::setup();
	print "startup complete";
}
```

For module functions and non-function symbols, unresolved names like `mymod::fn(...)` and `mymod::CONST` are checked when executed, not when parsed.

Use the CLI option to set the initial mode without editing scripts:

```sh
$ hawk --defermodsym on -f script.hawk
```

### @pragma striprecspc

When `FS` is a space-matching regex, this controls whether leading/trailing blank fields are removed.

- @pragma striprecspc on
```sh
$ echo '  a  b  c  d  ' | hawk '@pragma striprecspc on;
BEGIN { FS="[[:space:]]+"; }
{
    print "NF=" NF;
    for (i = 0; i < NF; i++) print i " [" $(i+1) "]";
}'
NF=4
0 [a]
1 [b]
2 [c]
3 [d]
```

- @pragma striprecspc off

``` sh
$ echo '  a  b  c  d  ' | hawk '@pragma striprecspc off;
BEGIN { FS="[[:space:]]+"; }
{
    print "NF=" NF;
    for (i = 0; i < NF; i++) print i " [" $(i+1) "]";
}'
NF=6
0 []
1 [a]
2 [b]
3 [c]
4 [d]
5 []
```

## @include and @include_once

`@include` inserts another file at parse time; the semicolon is optional. `@include_once` avoids duplicate inclusion.

```awk
function print_hello() { print "hello\n"; }
```

```awk
@include "hello.inc";
BEGIN { print_hello(); }
```

```awk
@include_once "hello.inc";
@include_once "hello.inc";
BEGIN { print_hello(); }
```

You can use them inside a block or at the top level:

```awk
BEGIN {
	@include "init.inc";
	...
}
```

## Comments

`Hawk` supports a single-line comment that begins with a hash sign # and the C-style multi-line comment.

```awk
x = y; # assign y to x.
/*
this line is ignored.
this line is ignored too.
*/
```

## Reserved Words

The following words are reserved and cannot be used as a variable name, a parameter name, or a function name.

 - @abort
 - @argc
 - @argv
 - @const
 - @false
 - @global
 - @include
 - @include_once
 - @local
 - @nil
 - @pragma
 - @reset
 - @true
 - BEGIN
 - END
 - break
 - case
 - continue
 - default
 - delete
 - do
 - else
 - exit
 - for
 - function
 - getbline
 - getline
 - if
 - in
 - next
 - nextfile
 - nextofile
 - print
 - printf
 - return
 - while
 - switch

However, some of these words not beginning with `@` can be used as normal names in the context of a module call. For example, `mymod::break`. In practice, the predefined names used for built-in commands, functions, and variables are treated as if they are reserved since you can't create another definition with the same name.

## Some Examples

- Print the first 10 even numbers
```awk
BEGIN {
	i = 0
	n = 1
	while (i < 10) {
		if (n % 2 == 0) {
			print n
			i++
		}
		n++
	}
}
```

- Prompt the user for a positive number
```awk
BEGIN {
	do {
		printf "Enter a positive number: "
		getline num
	} while (num <= 0)
	print "You entered:", num
}
```

- Print the multiplication table
```awk
BEGIN {
	for (i = 1; i <= 10; i++) {
		for (j = 1; j <= 10; j++) {
			printf "%4d", i * j
		}
		printf "\n"
	}
}
```

- Print only the even numbers from 1 to 16
```awk
BEGIN {
	for (i = 1; i <= 20; i++) {
		if (i % 2 != 0) {
			continue
		}
		print i
		if (i >= 16) {
			break
		}
	}
}
```

- Count the frequency of words in a file
```awk
{
	n = split($0, words, /[^[:alnum:]_]+/)
	for (i = 1; i <= n; i++) {
		freq[words[i]]++
	}
}

END {
	for (w in freq) {
		printf "%s: %d\n", w, freq[w]
	}
}
```

## Garbage Collection

The primary value management is reference counting based but `map` and `array` values are garbage-collected additionally.

## Modules

Hawk supports various modules.

### Hawk

- hawk::array
- hawk::call
- hawk::cmgr_exists
- hawk::function_exists
- hawk::gc
- hawk::gc_get_threshold
- hawk::gc_set_threshold
- hawk::gcrefs
- hawk::hash
- hawk::isarray
- hawk::ismap
- hawk::isnil
- hawk::map
- hawk::modlibdirs
- hawk::type
- hawk::typename
- hawk::GC_NUM_GENS

### String
The `str` module provides an extensive set of string manipulation functions.

- str::frombase64 - decode a base64-encoded byte string
- str::fromcharcode
- str::fromhex
- str::gsub - equivalent to gsub
- str::index
- str::isalnum
- str::isalpha
- str::isblank
- str::iscntrl
- str::isdigit
- str::isgraph
- str::islower
- str::isprint
- str::ispunct
- str::isspace
- str::isupper
- str::isxdigit
- str::length - equivalent to length
- str::ltrim
- str::match - similar to match. the optional third argument is the search start index. the optional fourth argument is equivalent to the third argument to match().
- str::normspace
- str::printf - equivalent to sprintf
- str::rindex
- str::rtrim
- str::split - equivalent to split
- str::sub - equivalent to sub
- str::substr - equivalent to substr
- str::tobase64 - encode data to a base64 byte string
- str::tocharcode - get the numeric value of the first character
- str::tohex
- str::tolower - equivalent to tolower
- str::tonum - convert a string to a number. a numeric value passed as a parameter is returned as it is. the leading prefix of `0b`, `0` or `0o`, and `0x` specifies the radix of 2, 8, 16 respectively. conversion stops when the end of the string is reached or the first invalid character for conversion is encountered.
- str::toupper - equivalent to toupper
- str::trim


### System

The `sys` module provides various functions concerning the underlying operation system.

- sys::basename
- sys::chmod
- sys::close
- sys::closedir
- sys::dirname
- sys::dup
- sys::errmsg
- sys::fork
- sys::getegid
- sys::getenv
- sys::geteuid
- sys::getgid
- sys::getpid
- sys::getppid
- sys::gettid
- sys::gettime
- sys::getuid
- sys::kill
- sys::mkdir
- sys::mktime
- sys::open
- sys::opendir
- sys::openfd
- sys::pipe
- sys::raise
- sys::read
- sys::readdir
- sys::setttime
- sys::signal
- sys::sleep
- sys::strftime
- sys::system
- sys::unlink
- sys::wait
- sys::write

#### Signals

Use these to register handlers and raise signals from Hawk.

- sys::signal(sig, func) sets a handler function for a signal number. pass nil to clear it.
- sys::raise(sig) raises a signal to the current runtime.
- sys::kill(pid, sig) sends a signal to another process or to self.

Example:

```awk
function on_int(sig) { print "got", sig }

BEGIN {
	sys::signal(sys::SIGINT, on_int);
	sys::raise(sys::SIGINT);
	## sys::kill(sys::getpid(), sys::SIGINT);
	sys::signal(sys::SIGINT, @nil);
}
```


#### Raw byte reads

You may read the file in raw bytes.

```awk
BEGIN {
	f = sys::open("/etc/sysctl.conf", sys::O_RDONLY);
	if (f >= 0) {
		while (sys::read(f, x, 10) > 0) printf (B"%s", x);
		sys::close(f);
	}
}
```

#### Wrap a file descriptor

You can map a raw file descriptor to a handle created by this module and use it.

```awk
BEGIN {
	a = sys::openfd(1);
	sys::write(a, B"let me write something here\n");
	sys::close(a, sys::C_KEEPFD); ## set C_KEEPFD to release 1 without closing it.
	##sys::close(a);
	print "done\n";
}
```

#### Pipe to a child process

Creating pipes and sharing them with a child process is not big an issue.

```awk
BEGIN {
	if (sys::pipe(p0, p1, sys::O_CLOEXEC | sys::O_NONBLOCK) <= -1)
	##if (sys::pipe(p0, p1, sys::O_CLOEXEC) <= -1)
	##if (sys::pipe(p0, p1) <= -1)
	{
		print "pipe error";
		return -1;
	}
	a = sys::fork();
	if (a <= -1) 
	{
		print "fork error";
		sys::close (p0);
		sys::close (p1);
	}
	else if (a == 0)
	{
		## child
		printf ("child.... %d %d %d\n", sys::getpid(), p0, p1);
		sys::close (p1);
		while (1)
		{
			n = sys::read (p0, k, 3);
			if (n <= 0) 
			{
				if (n == sys::RC_EAGAIN) continue; ## nonblock but data not available
				if (n != 0) print "ERROR: " sys::errmsg();
				break;
			}
			print k;
		}
		sys::close (p0);
		return 123;
	}
	else
	{
		## parent
		printf ("parent.... %d %d %d\n", sys::getpid(), p0, p1);
		sys::close (p0);
		sys::write (p1, B"hello");
		sys::write (p1, B"world");
		sys::close (p1);

		##sys::wait(a, status, sys::WNOHANG);
		while (sys::wait(a, status) != a);
		if (sys::WIFEXITED(status)) print "Exit code: " sys::WEXITSTATUS(status);
		else print "Child terminated abnormally"
	}
}
```

#### Read child stdout

You can read standard output of a child process in a parent process.

```awk
BEGIN {
	if (sys::pipe(p0, p1, sys::O_NONBLOCK | sys::O_CLOEXEC) <= -1)
	{
			print "pipe error";
			return -1;
	}
	a = sys::fork();
	if (a <= -1)
	{
		print "fork error";
		sys::close (p0);
		sys::close (p1);
	}
	else if (a == 0)
	{
		## child
		sys::close (p0);

		stdout = sys::openfd(1);
		sys::dup(p1, stdout);

		print B"hello world";
		print B"testing sys::dup()";
		print B"writing to standard output..";

		sys::close (p1);
		sys::close (stdout);
	}
	else
	{
		sys::close (p1);
		while (1)
		{
			n = sys::read(p0, k, 10);
			if (n <= 0)
			{
				if (n == sys::RC_EAGAIN) continue; ## nonblock but data not available
				if (n != 0) print "ERROR: " sys::errmsg();
				break;
			}
			print "[" k "]";
		}
		sys::close (p0);
		sys::wait(a);
	}
}
```

#### Duplicate file handles

You can duplicate file handles as necessary.

```awk
BEGIN {
	a = sys::open("/etc/inittab", sys::O_RDONLY);
	x = sys::open("/etc/fstab", sys::O_RDONLY);

	b = sys::dup(a);
	sys::close(a);

	while (sys::read(b, abc, 100) > 0) printf (B"%s", abc);

	print "-------------------------------";

	c = sys::dup(x, b, sys::O_CLOEXEC);
	## assertion: b == c
	sys::close (x);

	while (sys::read(c, abc, 100) > 0) printf (B"%s", abc);
	sys::close (c);
}
```

#### Directory traversal

Directory traversal is easy.

```awk
BEGIN {
	d = sys::opendir("/etc", sys::DIR_SORT);
	if (d >= 0)
	{
		while (sys::readdir(d,a) > 0)
		{
			print a;
			sys::stat("/etc/" %% a, b);
			for (i in b) print "\t", i, b[i];
		}
		sys::closedir(d);
	} 
}
```

#### Network interface info

You can get information of a network interface.

```awk
BEGIN { 
	if (sys::getnwifcfg("lo", sys::NWIFCFG_IN6, x) <= -1)
		print sys::errmsg();
	else
		for (i in x) print i, x[i]; 
}
```

#### Sockets

Socket functions are available.

```awk
BEGIN
{
	s = sys::socket();
	...
	sys::close (s);
}
```

### ffi

- ffi::open
- ffi::close
- ffi::call
- ffi::errmsg

```awk
BEGIN {
	ffi = ffi::open();
	if (ffi::call(ffi, r, @B"getenv", @B"s>s", "PATH") <= -1) print ffi::errmsg();
	else print r;
	ffi::close (ffi);
}
```

### mysql

```awk
BEGIN {
	mysql = mysql::open();

	if (mysql::connect(mysql, "localhost", "username", "password", "mysql") <= -1)
	{
		print "connect error -", mysql::errmsg();
	}

	if (mysql::query(mysql, "select * from user") <= -1)
	{
		print "query error -", mysql::errmsg();
	}

	result = mysql::store_result(mysql);
	if (result <= -1)
	{
		print "store result error - ", mysql::errmsg();
	}

	while (mysql::fetch_row(result, row) > 0)
	{
		ncols = length(row);
		for (i = 0; i < ncols; i++) print row[i];
		print "----";
	}

	mysql::free_result(result);

	mysql::close(mysql);
}
```

### sqlite
Assuming `/tmp/test.db` with the following schema,

```
sqlite> .schema
CREATE TABLE a(x int, y varchar(255));
```

You can retreive all rows as shown below:
```awk
@pragma entry main
@pragma implicit off

function main() {
	@local db, stmt, row, i, ncols;

	db = sqlite::open();
	if (db <= -1) {
		print "open error -", sqlite::errmsg();
		return;
	}

	if (sqlite::connect(db, "/tmp/test.db", sqlite::CONNECT_READWRITE) <= -1) {
		print "connect error -", sqlite::errmsg();
		sqlite::close(db);
		return;
	}

	sqlite::exec(db, "begin transaction");
	sqlite::exec(db, "delete from a");
	for (i = 0; i < 10; i++) {
		@local sql, fld;
		if (sqlite::escape_string(db, ((i % 2)? @b"'STXETX'": "'␂␃'") %% (math::rand() * 100), fld) <= -1) {
			print "escape_string error -", sqlite::errmsg();
			sqlite::exec(db, "rollback");
			sqlite::close(db);
			return;
		}
		sql=sprintf("insert into a(x,y) values(%d,'%s')", math::rand() * 100, fld);
		print sql;
		if (sqlite::exec(db, sql) <= -1) {
			print "exec error -", sqlite::errmsg();
			sqlite::exec(db, "rollback");
			sqlite::close(db);
			return;
		}
	}
	sqlite::exec(db, "commit");

	stmt = sqlite::prepare(db, "select x,y from a where x>?");
	if (stmt <= -1) {
		print "prepare error -", sqlite::errmsg();
		sqlite::close(db);
		return;
	}

	if (sqlite::bind(stmt, 1, 10) <= -1) {
		print "bind error -", sqlite::errmsg();
		sqlite::finalize(stmt);
		sqlite::close(db);
		return;
	}

	ncols = sqlite::column_count(stmt);
	printf ("TOTAL %d COLUMNS:\n", ncols);
	for (i = 1; i <= ncols; i++) {
		print "-", i, sqlite::column_name(stmt, i);
	}
	while (sqlite::fetch_row(stmt, row, sqlite::FETCH_ROW_ARRAY) > 0) {
		print "[id]", row[1], "[name]", row[2];
	}

	sqlite::finalize(stmt);
	sqlite::close(db);
}
```

## Incompatibility with AWK

### Parameter passing

In AWK, it is possible for the caller to pass an uninitialized variable as a function parameter and obtain a modified value if the called function sets it to an array.

```awk
function q(a) {
  a[1] = 20;
  a[2] = 30;
}

BEGIN {
  q(x);
  for (i in x)
    print i, x[i];
}
```

In Hawk, to achieve the same effect, you can indicate call-by-reference by prefixing the parameter name with an ampersand (&).

```awk
function q(&a) {
  a[1] = 20;
  a[2] = 30;
}

BEGIN {
  q(x);
  for (i in x)
    print i, x[i];
}
```

Alternatively, you may create an array or a map before passing it to a function.

```awk
function q(a) {
  a[1] = 20;
  a[2] = 30;
}

BEGIN {
  x[3] = 99; delete (x[3]);  ## x = hawk::array() or x = hawk::map() also will do
  q(x);
  for (i in x)
    print i, x[i];
}
```

### Positional variable expression

There are subtle differences in handling expressions for positional variables. In Hawk, many of the ambiguity issues can be resolved by enclosing the expression in parentheses.

| Expression   | Hawk          | AWK             |
|--------------|---------------|-----------------|
| `$++$++i`    | syntax error  | OK              |
| `$(++$(++i))`| OK            | syntax error    |

### Others
- `return` is allowed in `BEGIN` blocks, `END` blocks, and pattern-action blocks.