hyung-hwan/hawk
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

updated README.md

Browse Source
This commit is contained in:
hyung-hwan 2023-07-04 12:43:06 +09:00
parent 3f01a77f1e
commit 9fac7cfc98
1 changed files with 410 additions and 449 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

385
README.md
View File

@ -9,139 +9,35 @@
## Language <a name="language"></a>
Hawk is an AWK interpreter with many extended features implemetned by who 'H'
stands for. It aims to be an easy-to-embed implementation as well as used
standalone.
`Hawk` is an `AWK` interpreter with many extended features implemented by its creator, with 'H' representing the initial of the creator's name. It aims to be an easy-to-embed implementation as well as a standalone tool.
### Program Structure
At its core, `Hawk` largely supports all the fundamental features of `AWK`, ensuring compatibility with existing AWK programs and scripts, while introducing subtle differences in behavior, which will be explained in the [incompatibility](#incompatibility-with-awk) section. The following is an overview of the basic AWK features supported by `Hawk`:
A Hawk program is composed of the following elements at the top level.
1. Pattern-Action Statements: Hawk operates on a series of pattern-action statements. Each statement consists of a pattern that matches input records and an associated action that is executed when the pattern matches.
1. Record Processing: Hawk processes input data by splitting it into records and fields. Records are typically lines in a file, while fields are segments of each record separated by a field separator (by default, whitespace). This enables powerful text processing capabilities.
1. Built-in Variables: Hawk provides a set of built-in variables that facilitate data manipulation. Commonly used variables include NF (number of fields in the current record), NR (current record number), and $n (the value of the nth field in the current record).
1. Built-in Functions: Hawk offers a rich set of built-in functions to perform various operations on data. These functions include string manipulation, numeric calculations, regular expression matching, and more. You can harness their power to transform and analyze your input data effectively.
1. Output Formatting: Hawk provides flexible control over the formatting and presentation of output. You can customize the field and record separators, control the output field width, and apply formatting rules to align columns.
- pattern-action block pair
- BEGIN action block pair
- END action block pair
- action block without a pattern
- pattern without an action block
- user-defined function
- @global variable declaration
- @include directive
- @pragma directive
However, none of the above is mandatory. Hawk accepts an empty program.
### Pattern-Action Block Pair
A pattern-action pair is composed of a pattern and an action block as shown below:
pattern {
statement
statement
...
}
A pattern can be one of the followings when specified:
- expression
- first-expression, last-expression
- *BEGIN*
- *END*
An action block is a series of statements enclosed in a curly bracket pair. The *BEGIN* and *END* patterns require an action block while normal patterns don't. When no action block is specified for a normal pattern, it is treated
as if `{ print $0; }` is specified.
Hawk executes the action block for the *BEGIN* pattern when it starts executing a program; No start-up action is taken if no *BEGIN* pattern-action pair is specified. If a normal pattern-action pair and/or the *END*
pattern-action is specified, it reads the standard input stream. For each input line it reads, it checks if a normal pattern expression evaluates to true. For each pattern that evaluates to true, it executes the action block specified for
the pattern. When it reaches the end of the input stream, it executes the action block for the *END* pattern.
Hawk allows zero or more *BEGIN* patterns. When multiple *BEGIN* patterns are specified, it executes their action blocks in their appearance order in the program. The same applies to the *END* patterns and their action blocks. It
doesn't read the standard input stream for programs composed of BEGIN blocks only whereas it reads the stream as long as there is an action block for the END pattern or a normal pattern. It evaluates an empty pattern to true;
As a result, the action block for an empty pattern is executed for all input lines read.
You can compose a pattern range by putting 2 patterns separated by a comma. The pattern range evaluates to true once the first expression evaluates to true until the last expression evaluates to true.
The following code snippet is a valid Hawk program that prints the string *hello, world* to the console and exits.
BEGIN {
print "hello, world";
}
This program prints "hello, world" followed by "hello, all" to the console.
BEGIN {
print "hello, world";
}
BEGIN {
print "hello, all";
}
For the following text input,
abcdefgahijklmn
1234567890
opqrstuvwxyzabc
9876543210
this program
BEGIN { mr=0; my_nr=0; }
/abc/ { print "[" $0 "]"; mr++; }
{ my_nr++; }
END {
print "total records: " NR;
print "total records selfcounted: " my_nr;
print "matching records: " mr;
}
produces the output text like this:
[abcdefgahijklmn]
[opqrstuvwxyzabc]
total records: 4
total records selfcounted: 4
matching records: 2
See the table for the order of execution indicated by the number and the result
of pattern evaluation enclosed in parenthesis. The action block is executed if
the evaluation result is true.
| | START-UP | abcdefgahijklmn | 1234567890 | opqrstuvwxyzabc | 9876543210 | SHUTDOWN |
|-------------------------------------|----------|-----------------|------------|-----------------|------------|----------|
| `BEGIN { mr = 0; my_nr=0; }` | 1(true) | | | | | |
| `/abc/ { print "[" $0 "]"; mr++; }` | | 2(true) | 4(false) | 6(true) | 8(false) | |
| `{ my_nr++; }` | | 3(true) | 5(true) | 7(true) | 9(true) | |
| `END { print ... }` | | | | | | 10(true) |
For the same input, this program shows how to use a ranged pattern.
/abc/,/stu/ { print "[" $0 "]"; }
It produces the output text like this:
[abcdefgahijklmn]
[1234567890]
[opqrstuvwxyzabc]
The regular expression `/abc/` matches the first input line and `/stu/` matches the
third input line. So the range is true between the first input line and the
third input line inclusive.
With these foundational features, Hawk ensures compatibility with existing AWK scripts and enables you to utilize the vast range of AWK resources available.
### Entry Point
The typical execution begins with the BEGIN block, goes through pattern-action blocks, and eaches the END block. If you like to use a function as an entry point, you may set a function name with @pragma entry.
The typical execution begins with the `BEGIN` block, proceeds through pattern-action blocks, and concludes with the `END` block. If you would like to use a function as the entry point, you can specify a function name using `@pragma entry`.
@pragma entry main
```
@pragma entry main
function main ()
{
function main ()
{
print "hello, world";
}
}
```
### Pragmas
Besides the entry pragma, there are other prgrma available.
Besides the `entry` pragma, there are other prgrmas available.
A pragma item of the file scope can be placed in any source files.
A pragma item of the global scope can appear only once thoughout the all source files.
@ -151,51 +47,68 @@ A pragma item of the global scope can appear only once thoughout the all source
| implicit | file | on, off | allow undeclared variables |
| multilinestr | file | on, off | allow a multiline string literal without continuation |
| entry | global | function name | change the program entry point |
| striprecspc | global | on, off | |
| striprecspc | global | on, off | trim leading and trailing spaces when convering a string to a number |
```
@pragma implicit off
BEGIN { a = 10; } ## syntax error - undefined identifier 'a'
```
```
@pragma implicit off
BEGIN { @local a; a = 10; } # syntax ok - a is declared before use.
```
### @include and @include_once
The @include directive inserts the contents of the file specified in the following string as if they appeared in the source stream being processed.
Assuming the hello.inc file contains the print_hello() function as shown below,
Assuming the `hello.inc` file contains the print_hello() function as shown below,
function print_hello() { print "hello\n"; }
```
function print_hello() { print "hello\n"; }
```
You may include the the file and use the function.
@include "hello.inc";
BEGIN { print_hello(); }
```
@include "hello.inc";
BEGIN { print_hello(); }
```
The semicolon after the included file name is optional. You could write @include "hello.inc" without the ending semicolon.
@include_once is similar to @include except it doesn't include the same file multiple times.
@include_once "hello.inc";
@include_once "hello.inc";
BEGIN { print_hello(); }
```
@include_once "hello.inc";
@include_once "hello.inc";
BEGIN { print_hello(); }
```
In this example, print_hello() is not included twice.
In this example, `print_hello()` is not included twice.
You may use @include and @include_once inside a block as well as at the top level.
BEGIN {
```
BEGIN {
@include "init.inc";
...
}
}
```
### Comments
Hawk supports a single-line commnt that begins with a hash sign # and the C-style multi-line comment.
`Hawk` supports a single-line commnt that begins with a hash sign # and the C-style multi-line comment.
x = y; # assign y to x.
/*
this line is ignored.
this line is ignored too.
*/
```
x = y; # assign y to x.
/*
this line is ignored.
this line is ignored too.
*/
```
### Reserved Words
@ -239,26 +152,30 @@ However, these words can be used as normal names in the context of a module call
- floating-point number
- string
- byte string
- array
- array - light-weight array with numeric index only
- map - conventional AWK array
- function
- regular expression
To know the current type of a value, call typename().
To know the current type of a value, call `hawk::typename()`.
function f() { return 10; }
BEGIN {
```
function f() { return 10; }
BEGIN {
a="hello";
b=12345;
print hawk::typename(a), hawk::typename(b), hawk::typename(c), hawk::typename(f), hawk::typename(1.23), hawk::typename(B"world");
}
}
```
A regular expression literal is special in that it never appears as an indendent value and still entails a match operation against $0 without an match operator.
BEGIN { $0="ab"; print /ab/, typename(/ab/); }
```
BEGIN { $0="ab"; print /ab/, hawk::typename(/ab/); }
```
For this reason, there is no way to get the type name of a regular expressin literal.
### Numbers ###
An integer begins with a numeric digit between 0 and 9 inclusive and can be
@ -269,29 +186,29 @@ can be followed by e or E, and optionally a series of numeric digits with a
optional single sign letter. A floating-point number may begin with a floting
point with a preceeding number.
369 # integer
3.69 # floating-pointe number
13. # 13.0
.369 # 0.369
34e-2 # 34 * (10 ** -2)
34e+2 # 34 * (10 ** 2)
34.56e # 34.56
34.56E3
- 369 # integer
- 3.69 # floating-pointe number
- 13. # 13.0
- .369 # 0.369
- 34e-2 # 34 * (10 ** -2)
- 34e+2 # 34 * (10 ** 2)
- 34.56e # 34.56
- 34.56E3
An integer can be prefixed with 0x, 0, 0b for a hexa-decimal number, an octal
number, and a binary number respectively. For a hexa-decimal number, letters
from A to F can form a number case-insenstively in addition to numeric digits.
0xA1 # 161
0xB0b0 # 45232
020 # 16
0b101 # 5
- 0xA1 # 161
- 0xB0b0 # 45232
- 020 # 16
- 0b101 # 5
If the prefix is not followed by any numeric digits, it is still a valid token
and represents the value of 0.
0x # 0x0 but not desirable.
0b # 0b0 but not desirable.
- 0x # 0x0 but not desirable.
- 0b # 0b0 but not desirable.
### Modules <a name="modules"></a>
@ -373,27 +290,31 @@ The *sys* module provides various functions concerning the underlying operation
You may read the file in raw bytes.
BEGIN {
```
BEGIN {
f = sys::open("/etc/sysctl.conf", sys::O_RDONLY);
while (sys::read(f, x, 10) > 0) printf (B"%s", x);
sys::close (f);
}
}
```
You can map a raw file descriptor to a handle created by this module and use it.
BEGIN {
```
BEGIN {
a = sys::openfd(1);
sys::write (a, B"let me write something here\n");
sys::close (a, sys::C_KEEPFD); ## set C_KEEPFD to release 1 without closing it.
##sys::close (a);
print "done\n";
}
}
```
Creating pipes and sharing them with a child process is not big an issue.
BEGIN {
```
BEGIN {
if (sys::pipe(p0, p1, sys::O_CLOEXEC | sys::O_NONBLOCK) <= -1)
##if (sys::pipe(p0, p1, sys::O_CLOEXEC) <= -1)
##if (sys::pipe(p0, p1) <= -1)
@ -441,12 +362,13 @@ Creating pipes and sharing them with a child process is not big an issue.
if (sys::WIFEXITED(status)) print "Exit code: " sys::WEXITSTATUS(status);
else print "Child terminated abnormally"
}
}
}
```
You can read standard output of a child process in a parent process.
BEGIN {
```
BEGIN {
if (sys::pipe(p0, p1, sys::O_NONBLOCK | sys::O_CLOEXEC) <= -1)
{
print "pipe error";
@ -491,11 +413,13 @@ You can read standard output of a child process in a parent process.
sys::close (p0);
sys::wait(a);
}
}
}
```
You can duplicate file handles as necessary.
BEGIN {
```
BEGIN {
a = sys::open("/etc/inittab", sys::O_RDONLY);
x = sys::open("/etc/fstab", sys::O_RDONLY);
@ -512,12 +436,13 @@ You can duplicate file handles as necessary.
while (sys::read(c, abc, 100) > 0) printf (B"%s", abc);
sys::close (c);
}
}
```
Directory traversal is easy.
BEGIN {
```
BEGIN {
d = sys::opendir("/etc", sys::DIR_SORT);
if (d >= 0)
{
@ -529,34 +454,38 @@ Directory traversal is easy.
}
sys::closedir(d);
}
}
}
```
You can get information of a network interface.
BEGIN {
```
BEGIN {
if (sys::getnwifcfg("lo", sys::NWIFCFG_IN6, x) <= -1)
print sys::errmsg();
else
for (i in x) print i, x[i];
}
}
```
Socket functions are available.
BEGIN
{
```
BEGIN
{
s = sys::socket();
...
sys::close (s);
}
}
```
### ffi
### mysql
BEGIN {
```
BEGIN {
mysql = mysql::open();
if (mysql::connect(mysql, "localhost", "username", "password", "mysql") <= -1)
@ -585,39 +514,68 @@ Socket functions are available.
mysql::free_result(result);
mysql::close(mysql);
}
}
```
### Incompatibility with AWK
### Incompatibility with AWK <a name="incompatibility-with-awk"></a>
#### Parameter passing
In AWK, the caller can pass an uninitialized variable as a function parameter and get a changed value if the callled function sets it to an array.
In AWK, it is possible for the caller to pass an uninitialized variable as a function parameter and obtain a modified value if the called function sets it to an array.
```
function q(a) {
a[1] = 20;
a[2] = 30;
}
function q(a) {a[1]=20; a[2]=30;}
BEGIN { q(x); for (i in x) print i, x[i]; }
BEGIN {
q(x);
for (i in x)
print i, x[i];
}
```
In Hawk, you can prefix the pramater name with & to indicate call-by-reference for the same effect.
In Hawk, to achieve the same effect, you can indicate call-by-reference by prefixing the parameter name with an ampersand (&).
function q(&a) {a[1]=20; a[2]=30;}
BEGIN { q(x); for (i in x) print i, x[i]; }
```
function q(&a) {
a[1] = 20;
a[2] = 30;
}
BEGIN {
q(x);
for (i in x)
print i, x[i];
}
```
Alternatively, you may form an array before passing it to a function.
Alternatively, you may create an array or a map before passing it to a function.
function q(a) {a[1]=20; a[2]=30;}
BEGIN { x[3]=99; q(x); for (i in x) print i, x[i]; }'
```
function q(a) {
a[1] = 20;
a[2] = 30;
}
BEGIN {
x[3] = 99; delete (x[3]); ## x = hawk::array() or x = hawk::map() also will do
q(x);
for (i in x)
print i, x[i];
}
```
### Positional variable expression
There are subtle differences when you put an expression for a position variable. In Hawk, most of the ambiguity issues are resolved if you enclose the expression inside parentheses.
There are subtle differences in handling expressions for positional variables. In Hawk, many of the ambiguity issues can be resolved by enclosing the expression in parentheses.
| Expression | HAWK | AWK |
|--------------|---------------|-----------------|
| $++$++i | syntax error | OK |
| $(++$(++i)) | OK | syntax error |
| `$++$++i` | syntax error | OK |
| `$(++$(++i))`| OK | syntax error |
## Embedding Guide <a name="embedding-guide"></a>
@ -631,13 +589,14 @@ To use hawk in your program, do the followings:
- destroy the runtime context
- destroy the hawk instance
The following sample illustrates the basic steps hightlighed above.
The following sample illustrates the basic steps highlighted above.
#include <hawk-std.h>
#include <stdio.h>
#include <string.h>
```
#include <hawk-std.h>
#include <stdio.h>
#include <string.h>
static const hawk_bch_t* src =
static const hawk_bch_t* src =
"BEGIN {"
" for (i=2;i<=9;i++)"
" {"
@ -647,8 +606,8 @@ The following sample illustrates the basic steps hightlighed above.
" }"
"}";
int main ()
{
int main ()
{
hawk_t* hawk = HAWK_NULL;
hawk_rtx_t* rtx = HAWK_NULL;
hawk_val_t* retv;
@ -703,20 +662,21 @@ The following sample illustrates the basic steps hightlighed above.
hawk_rtx_refdownval (rtx, retv);
ret = 0;
oops:
oops:
if (rtx) hawk_rtx_close (rtx); /* destroy the runtime context */
if (hawk) hawk_close (hawk); /* destroy the hawk instance */
return -1;
}
}
```
If you prefer C++, you may use the Hawk/HawkStd wrapper classes to simplify the task. The C++ classes are inferior to the C equivalents in that they don't allow creation of multiple runtime contexts over a single hawk instance. Here is the sample code that prints "hello, world".
#include <Hawk.hpp>
#include <stdio.h>
```
#include <Hawk.hpp>
#include <stdio.h>
int main ()
{
int main ()
{
HAWK::HawkStd hawk;
if (hawk.open() <= -1)
@ -738,4 +698,5 @@ If you prefer C++, you may use the Hawk/HawkStd wrapper classes to simplify the
hawk.close ();
return 0;
}
}
```