💾 Archived View for spam.works › users › emery › why-nim.gmi captured on 2023-06-14 at 14:33:02. Gemini links have been rewritten to link to archived content

View Raw

More Information

-=-=-=-=-=-=-

Why Nim?

I was asked to put together a document on why I recommend the Nim language, so here follows at brief overview of my problems with the language, what I like, and some code examples. I may update this without notice.

Disadvantages

Dependency management

Nim packages can be published using the Nimble package manager. My issues with Nimble are straightforward:

• Packages can be published un-versioned
• Dependencies can be specified with version constraints, but cannot be locked to specific commits.
• Messy code, and poor error messages. The Nimble implementation is hacky and errors seem to propagate to unrelated code with bad error handling.

I have made some effort at managing Nimble using Nix, but it's always been too fragile to work consistenly.

https://github.com/nix-community/flake-nimble

Language features

The language has many keywords, compiler pragmas, and a flexible grammer. This can be overwhelming when writting code and complicates code comprehension. Most of the features can be ignored without safety or perfomance costs, so it shouldn't be a problem for a disciplined developer.

Advantages

Compiles to C and C++

I started using Nim because I was maintaining a parts of large C++ codebase with a C++ API, but I also recognize that C++ is a deeply flawed language. Compiling to C++ allowed me to easily wrap and use these C++ ABIs. Memory management is concern when passing pointers across the threshold, but so far it hasn't been a hassle.

Wrapping C:

{.passC: staticExec("pkg-config --cflags opus libopusenc").}
{.passL: staticExec("pkg-config --libs opus libopusenc").}

{.pragma: opeHeader, header: "opus/opusenc.h".}
{.pragma: opeProc, opeHeader, importc: "ope_$1".}

Type
  OpeEncoder = distinct pointer
  Encoder* = object
    ## Opaque encoder object.
    ope: OpeEncoder

type OpusEncError* = object of CatchableError

template opeCheck(body: untyped) =
  # Template for checking ``res`` and ``err`` values after ``body``.
  var
    res {.inject.}: cint
    err {.inject.} = addr res
  body
  if res != OK:
    raise newException(OpusEncError, $strerror(res))

# ...

proc initFileEncoder*(
    path: string; com: Comments; rate: int32; channels: cint; family = monoStereo): Encoder =
  ## Create a new OggOpus file.
  proc encoder_create_file(path: cstring; comments: OpeComments;
                               rate: int32; channels: cint; family: cint;
                               error: ptr cint): OpeEncoder {.opeProc.}
  opeCheck:
    result.ope = encoder_create_file(
      path, com.ope, (int32)rate, channels, (cint)family, err)

proc `=destroy`*(enc: var Encoder) =
  ## Destructor for object holding malloc'ed C pointer.
  if not enc.ope.pointer.isNil:
    proc encoder_destroy(enc: OpeEncoder) {.opeProc.}
    encoder_destroy(enc.ope)
    reset enc.ope

proc write*(enc: Encoder; pcm: ptr int16; samplesPerChannel: int) =
  ## Add/encode any number of 16-bit linear samples to the stream.
  proc encoder_write(enc: OpeEncoder; pcm: ptr int16;
                         samples_per_channel: cint): cint {.opeProc.}
  opeCheck:
    res = encoder_write(enc.ope, pcm, (cint)samplesPerChannel)

Opusenc wrapper

Wrapping the Genode C++ API:

const terminalH = "<terminal_session/connection.h>"

type
  ConnectionBase {.
    importcpp: "Terminal::Connection", header: terminalH.} = object
  Connection = Constructible[ConnectionBase]

  TerminalClient* = ref TerminalClientObj
  TerminalClientObj = object
    conn: Connection
    readAvailSigh, sizeChangedSigh: SignalDispatcher

proc construct(c: Connection; env: GenodeEnv; label: cstring) {.
  importcpp: "#.construct(*#, @)", tags: [RpcEffect].}

proc read(c: Connection; buf: pointer; bufLen: int): int {.tags: [RpcEffect],
  importcpp: "#->read(@)".}

proc write(c: Connection; buf: cstring|pointer; bufLen: int): int {.tags: [RpcEffect],
  importcpp: "#->write(@)".}

# ...

proc newTerminalClient*(env: GenodeEnv; label = ""): TerminalClient=
  ## Open a new **Terminal** session.
  new result
  result.conn.construct(env, label)

proc read*(tc: TerminalClient; buffer: pointer; bufLen: int): int  =
  ## Read any available data from the terminal.
  tc.conn.read(buffer, bufLen)

proc write*(tc: TerminalClient; s: string): int  =
  ## Write a string to the terminal.
  tc.conn.write(s.cstring, s.len)

Genode Terminal wrapper

Portable language runtime

The language runtime is fairly easy to port to modern (post-1970s) environments. The garbage collector is freestanding and easy to interface to native page allocators. Freestanding thread-local storage emulation is also available. Bare-metal is supported, but I haven't tried it.

Robust type system

The type system makes expressing array types, type templates, and distinct types easy. The language allows values to be passed around easily regardless if they are heap or stack values, with good mutability tracking.

An example of generating fixed array types along with procedure for converting to and from string encodings:

template toxArrayType(T, N: untyped) =
  ## Generate an array type definition with hex conversions.
  type T* = object
    bytes*: array[N, uint8]
  func `







*(x: T): string =
    result = newStringOfCap(N*2)
    for b in x.bytes:
      result.add b.toHex
  func `to T`*(s: string): T =
    doAssert(s.len == N*2)
    let raw = parseHexStr s
    for i in 0..<N:
      result.bytes[i] = (uint8)raw[i]

toxArrayType(PublicKey, TOX_PUBLIC_KEY_SIZE)
toxArrayType(SecretKey, TOX_SECRET_KEY_SIZE)
toxArrayType(Address, TOX_ADDRESS_SIZE)
toxArrayType(Hash, TOX_HASH_LENGTH)
toxArrayType(ConferenceId, TOX_CONFERENCE_ID_SIZE)
toxArrayType(ConferenceUid, TOX_CONFERENCE_UID_SIZE)

Tox wrapper

Not object oriented

The object types are easy to understand. Object inheritance is only applicable to object members, and only one level deep. There are no object "methods", access to object memembers can be restricted to the local module or allowed globally. Procedures can be added and overriden for objects at arbitrary locations. Due to the common influence of Oberon the objects are similar to those of Go. Object variants, also known as tagged unions, are one of my favorite features, and can be used in place of C++ inheritance or Go interface types.

An example of modeling multiple CBOR item types using a common object, the "kind" field desciminates which fields are in use for an object instance:

type
  CborNodeKind* = enum
    cborUnsigned = 0,
    cborNegative = 1,
    cborBytes = 2,
    cborText = 3,
    cborArray = 4,
    cborMap = 5,
    cborTag = 6,
    cborSimple = 7,
    cborFloat,
    cborRaw

  CborNode* = object
    tag: Option[uint64]
    case kind*: CborNodeKind
    of cborUnsigned:
      uint*: uint64
    of cborNegative:
      int*: int64
    of cborBytes:
      bytes*: seq[byte]
    of cborText:
      text*: string
    of cborArray:
      seq*: seq[CborNode]
    of cborMap:
      map*: OrderedTable[CborNode, CborNode]
    of cborTag:
      discard
    of cborSimple:
      simple*: uint8
    of cborFloat:
      float*: float64
    of cborRaw:
      raw*: string

CBOR

Side effect tracking

The language distinguishes between procudures and functions. Functions are a subset of producers where side effects are restricted, no I/O or other changes to global state may be made. This is helpful for implementing functional datastructures or deterministic procedures.

Deterministic cryptographic signing, the library has no side-effects and relies on an entropy gathering callback:

type
  RandomBytes* = proc(buf: pointer; size: int)
    ## Procedure type for collecting entropy during
    ## key generation and signing. Please supply
    ## a procedure that writes `size` random bytes to `buf`.

func sign*(pair: KeyPair; msg: string|openArray[byte]|seq[byte]; rand: RandomBytes): string =
  ## Generate a SPHINCS⁺ signature. The passed `rand` procedure is used to
  ## create non-deterministic signatures which are generally recommended.
  var optRand: Nbytes
  rand(optRand.addr, n)
  pair.sign(msg, optRand)

proc generateKeypair*(seedProc: RandomBytes): KeyPair {.noSideEffect.} =
  ## Generate a SPHINCS⁺ key pair.
  seedProc(result.addr, n*3)
    # Randomize the seeds and PRF
  result.pk.root = ht_PKgen(result.sk, result.pk)
    # Compute root node of top-most subtree

SPHINCS⁺