Strings & bytes

This page shows how text and raw bytes work in d/Python. Text (str) is Unicode, so it can hold any character from any language. Raw bytes (bytes) are plain numbers from 0 to 255. d/Python keeps the two apart exactly as Python 3 does. After this page you will know which string methods you can use today, how to turn text into bytes, and which pieces are still missing.

Underneath, the d/OS runtime (the program that runs your compiled code) stores both kinds of value in the same kind of slot, but your program never sees that. len() of a str counts characters (code points); len() of a bytes value counts bytes; and mixing the two in one expression is an error, never a silent conversion.

Text

name = "Zoë"
print(len(name), name[0], name[-1], name[1:], name[::-1])
print("Hello, " + name + "!", name == "Zoë", "abc" < "abd", "ë" in name)
print(f"{name!r} has {len(name)} code points; {2 ** 10} bytes is 1 KiB")
print(str(42) + "!", repr("it's"), ascii("café"))

f-strings can insert any value that print can show, and the !r and !a conversions work. Format specifications inside the braces, such as {x:>8}, are not yet supported, and neither are % formatting and str.format; the compiler stops at that line and tells you.

The methods that are there

text = "  Small programs, useful programs.  "
print(text.strip(), text.lstrip(), text.rstrip(".  "))
print(text.split(), "a,b,,c".split(","), "a b c".rsplit(" ", 1))
print("-".join(["x", "y", "z"]), "abc".replace("b", "B"))
print("hello".find("l"), "hello".count("l"), "hello".startswith("he"), "hello".endswith("lo"))
print("key=value".partition("="), "a.b.c".rpartition("."))

Searching and counting, checking how a string starts and ends, partition and rpartition, trimming, splitting from either end, replace and join all work, for text and separately for bytes, with Python's rules for bounds and for what counts as Unicode whitespace. Changing case (upper, lower), the isdigit family, repeating a string with *, and formatting are not there yet; if you use one, the compiler stops and names the method that is missing.

Characters and code points

print(ord("A"), ord("🐍"), chr(65), chr(0x1F40D))
try:
    chr(-1)
except ValueError as error:
    print("ValueError:", error)
for letter in "abc":
    print(letter, ord(letter))

ord takes a one-character string, or a single byte, and gives its number. chr builds the character for an integer (or a boolean, which counts as 0 or 1); a wrong type raises TypeError and a number out of range raises ValueError, and you can catch both. Surrogate code points (a reserved range of Unicode that ordinary text never contains) cannot be stored in d/Python's strings yet, so they raise NotImplementedError.

Bytes

data = "héllo".encode()
print(data, len(data), data[1], data[0:2], b"\xc3" in data)
print(b"abc" + b"def", b"abc" == b"abc", b"abc"[0])
print(repr(b"a\x00b"), ascii(b"\xff"))

text.encode() gives you the UTF-8 bytes of a string. Behind the scenes it shares the string's storage rather than copying it, but what you get is a real bytes value with its own length, indexing and repr. You can call encode() with no arguments, or name UTF-8 as a literal by one of its usual spellings, with errors="strict". Other encodings, and decoding bytes back into text, are not yet supported.

Byte strings from d/OS

When you call into d/OS itself, or into a shared library file, .dbl, and get a string back, d/Python hands it to you as bytes, because those values can carry any binary data, not only text. You can compare them, join them with + and take their len(). Decoding them into Unicode text is not yet supported. See The dos Module.

Capacity

A single string, or a single entry in the program's shared string storage, can hold at most 65,535 bytes. This limit comes from the compiled program file, .dbc, not from d/Python itself. If a program goes over it, d/OS stops the program and reports the limit; it never quietly gives a shortened or different answer. Indexing a bytes value takes a fast path instead of the general slicing code, but Python's bounds checks and errors apply all the way up to that limit.

What the tests cover

Every claim on this page comes from a test program that runs in both d/Python and CPython and must print the same thing. The corpus page lists those programs, with their reviewed output.