💾 Archived View for tris.fyi › pydoc › codecs captured on 2023-04-26 at 13:27:48. Gemini links have been rewritten to link to archived content
⬅️ Previous capture (2023-01-29)
-=-=-=-=-=-=-
codecs -- Python Codec Registry, API and helpers. Written by Marc-Andre Lemburg (mal@lemburg.com). (c) Copyright CNRI, All Rights Reserved. NO WARRANTY.
This subclass of IncrementalDecoder can be used as the baseclass for an
incremental decoder if the decoder must be able to handle incomplete
byte sequences.
decode(self, input, final=False)
getstate(self)
reset(self)
setstate(self, state)
This subclass of IncrementalEncoder can be used as the baseclass for an
incremental encoder if the encoder must keep some of the output in a
buffer between calls to encode().
encode(self, input, final=False)
getstate(self)
reset(self)
setstate(self, state)
Defines the interface for stateless encoders/decoders.
The .encode()/.decode() methods may use different error
handling schemes by providing the errors argument. These
string values are predefined:
'strict' - raise a ValueError error (or a subclass)
'ignore' - ignore the character and continue with the next
'replace' - replace with a suitable replacement character;
Python will use the official U+FFFD REPLACEMENT
CHARACTER for the builtin Unicode codecs on
decoding and '?' on encoding.
'surrogateescape' - replace with private code points U+DCnn.
'xmlcharrefreplace' - Replace with the appropriate XML
character reference (only for encoding).
'backslashreplace' - Replace with backslashed escape sequences.
'namereplace' - Replace with \N{...} escape sequences
(only for encoding).
The set of allowed values can be extended via register_error.
decode(self, input, errors='strict')
Decodes the object input and returns a tuple (output
object, length consumed).
input must be an object which provides the bf_getreadbuf
buffer slot. Python strings, buffer objects and memory
mapped files are examples of objects providing this slot.
errors defines the error handling to apply. It defaults to
'strict' handling.
The method may not store state in the Codec instance. Use
StreamReader for codecs which have to keep state in order to
make decoding efficient.
The decoder must be able to handle zero length input and
return an empty object of the output object type in this
situation.
encode(self, input, errors='strict')
Encodes the object input and returns a tuple (output
object, length consumed).
errors defines the error handling to apply. It defaults to
'strict' handling.
The method may not store state in the Codec instance. Use
StreamWriter for codecs which have to keep state in order to
make encoding efficient.
The encoder must be able to handle zero length input and
return an empty object of the output object type in this
situation.
Codec details when looking up the codec registry
count(self, value, /) Return number of occurrences of value.
index(self, value, start=0, stop=9223372036854775807, /) Return first index of value. Raises ValueError if the value is not present.
An IncrementalDecoder decodes an input in multiple steps. The input can
be passed piece by piece to the decode() method. The IncrementalDecoder
remembers the state of the decoding process between calls to decode().
decode(self, input, final=False)
Decode input and returns the resulting object.
getstate(self)
Return the current state of the decoder.
This must be a (buffered_input, additional_state_info) tuple.
buffered_input must be a bytes object containing bytes that
were passed to decode() that have not yet been converted.
additional_state_info must be a non-negative integer
representing the state of the decoder WITHOUT yet having
processed the contents of buffered_input. In the initial state
and after reset(), getstate() must return (b"", 0).
reset(self)
Reset the decoder to the initial state.
setstate(self, state)
Set the current state of the decoder.
state must have been returned by getstate(). The effect of
setstate((b"", 0)) must be equivalent to reset().
An IncrementalEncoder encodes an input in multiple steps. The input can
be passed piece by piece to the encode() method. The IncrementalEncoder
remembers the state of the encoding process between calls to encode().
encode(self, input, final=False)
Encodes input and returns the resulting object.
getstate(self)
Return the current state of the encoder.
reset(self)
Resets the encoder to the initial state.
setstate(self, state)
Set the current state of the encoder. state must have been
returned by getstate().
str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.__str__() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.
capitalize(self, /) Return a capitalized version of the string. More specifically, make the first character have upper case and the rest lower case.
casefold(self, /) Return a version of the string suitable for caseless comparisons.
center(self, width, fillchar=' ', /) Return a centered string of length width. Padding is done using the specified fill character (default is a space).
count(...) S.count(sub[, start[, end]]) -> int Return the number of non-overlapping occurrences of substring sub in string S[start:end]. Optional arguments start and end are interpreted as in slice notation.
encode(self, /, encoding='utf-8', errors='strict')
Encode the string using the codec registered for encoding.
encoding
The encoding in which to encode the string.
errors
The error handling scheme to use for encoding errors.
The default is 'strict' meaning that encoding errors raise a
UnicodeEncodeError. Other possible values are 'ignore', 'replace' and
'xmlcharrefreplace' as well as any other name registered with
codecs.register_error that can handle UnicodeEncodeErrors.
endswith(...) S.endswith(suffix[, start[, end]]) -> bool Return True if S ends with the specified suffix, False otherwise. With optional start, test S beginning at that position. With optional end, stop comparing S at that position. suffix can also be a tuple of strings to try.
expandtabs(self, /, tabsize=8) Return a copy where all tab characters are expanded using spaces. If tabsize is not given, a tab size of 8 characters is assumed.
find(...) S.find(sub[, start[, end]]) -> int Return the lowest index in S where substring sub is found, such that sub is contained within S[start:end]. Optional arguments start and end are interpreted as in slice notation. Return -1 on failure.
format(...)
S.format(*args, **kwargs) -> str
Return a formatted version of S, using substitutions from args and kwargs.
The substitutions are identified by braces ('{' and '}').
format_map(...)
S.format_map(mapping) -> str
Return a formatted version of S, using substitutions from mapping.
The substitutions are identified by braces ('{' and '}').
index(...) S.index(sub[, start[, end]]) -> int Return the lowest index in S where substring sub is found, such that sub is contained within S[start:end]. Optional arguments start and end are interpreted as in slice notation. Raises ValueError when the substring is not found.
isalnum(self, /) Return True if the string is an alpha-numeric string, False otherwise. A string is alpha-numeric if all characters in the string are alpha-numeric and there is at least one character in the string.
isalpha(self, /) Return True if the string is an alphabetic string, False otherwise. A string is alphabetic if all characters in the string are alphabetic and there is at least one character in the string.
isascii(self, /) Return True if all characters in the string are ASCII, False otherwise. ASCII characters have code points in the range U+0000-U+007F. Empty string is ASCII too.
isdecimal(self, /) Return True if the string is a decimal string, False otherwise. A string is a decimal string if all characters in the string are decimal and there is at least one character in the string.
isdigit(self, /) Return True if the string is a digit string, False otherwise. A string is a digit string if all characters in the string are digits and there is at least one character in the string.
isidentifier(self, /) Return True if the string is a valid Python identifier, False otherwise. Call keyword.iskeyword(s) to test whether string s is a reserved identifier, such as "def" or "class".
islower(self, /) Return True if the string is a lowercase string, False otherwise. A string is lowercase if all cased characters in the string are lowercase and there is at least one cased character in the string.
isnumeric(self, /) Return True if the string is a numeric string, False otherwise. A string is numeric if all characters in the string are numeric and there is at least one character in the string.
isprintable(self, /) Return True if the string is printable, False otherwise. A string is printable if all of its characters are considered printable in repr() or if it is empty.
isspace(self, /) Return True if the string is a whitespace string, False otherwise. A string is whitespace if all characters in the string are whitespace and there is at least one character in the string.
istitle(self, /) Return True if the string is a title-cased string, False otherwise. In a title-cased string, upper- and title-case characters may only follow uncased characters and lowercase characters only cased ones.
isupper(self, /) Return True if the string is an uppercase string, False otherwise. A string is uppercase if all cased characters in the string are uppercase and there is at least one cased character in the string.
join(self, iterable, /) Concatenate any number of strings. The string whose method is called is inserted in between each given string. The result is returned as a new string. Example: '.'.join(['ab', 'pq', 'rs']) -> 'ab.pq.rs'
ljust(self, width, fillchar=' ', /) Return a left-justified string of length width. Padding is done using the specified fill character (default is a space).
lower(self, /) Return a copy of the string converted to lowercase.
lstrip(self, chars=None, /) Return a copy of the string with leading whitespace removed. If chars is given and not None, remove characters in chars instead.
maketrans(...) Return a translation table usable for str.translate(). If there is only one argument, it must be a dictionary mapping Unicode ordinals (integers) or characters to Unicode ordinals, strings or None. Character keys will be then converted to ordinals. If there are two arguments, they must be strings of equal length, and in the resulting dictionary, each character in x will be mapped to the character at the same position in y. If there is a third argument, it must be a string, whose characters will be mapped to None in the result.
partition(self, sep, /) Partition the string into three parts using the given separator. This will search for the separator in the string. If the separator is found, returns a 3-tuple containing the part before the separator, the separator itself, and the part after it. If the separator is not found, returns a 3-tuple containing the original string and two empty strings.
removeprefix(self, prefix, /) Return a str with the given prefix string removed if present. If the string starts with the prefix string, return string[len(prefix):]. Otherwise, return a copy of the original string.
removesuffix(self, suffix, /) Return a str with the given suffix string removed if present. If the string ends with the suffix string and that suffix is not empty, return string[:-len(suffix)]. Otherwise, return a copy of the original string.
replace(self, old, new, count=-1, /)
Return a copy with all occurrences of substring old replaced by new.
count
Maximum number of occurrences to replace.
-1 (the default value) means replace all occurrences.
If the optional argument count is given, only the first count occurrences are
replaced.
rfind(...) S.rfind(sub[, start[, end]]) -> int Return the highest index in S where substring sub is found, such that sub is contained within S[start:end]. Optional arguments start and end are interpreted as in slice notation. Return -1 on failure.
rindex(...) S.rindex(sub[, start[, end]]) -> int Return the highest index in S where substring sub is found, such that sub is contained within S[start:end]. Optional arguments start and end are interpreted as in slice notation. Raises ValueError when the substring is not found.
rjust(self, width, fillchar=' ', /) Return a right-justified string of length width. Padding is done using the specified fill character (default is a space).
rpartition(self, sep, /) Partition the string into three parts using the given separator. This will search for the separator in the string, starting at the end. If the separator is found, returns a 3-tuple containing the part before the separator, the separator itself, and the part after it. If the separator is not found, returns a 3-tuple containing two empty strings and the original string.
rsplit(self, /, sep=None, maxsplit=-1)
Return a list of the substrings in the string, using sep as the separator string.
sep
The separator used to split the string.
When set to None (the default value), will split on any whitespace
character (including \\n \\r \\t \\f and spaces) and will discard
empty strings from the result.
maxsplit
Maximum number of splits (starting from the left).
-1 (the default value) means no limit.
Splitting starts at the end of the string and works to the front.
rstrip(self, chars=None, /) Return a copy of the string with trailing whitespace removed. If chars is given and not None, remove characters in chars instead.
split(self, /, sep=None, maxsplit=-1)
Return a list of the substrings in the string, using sep as the separator string.
sep
The separator used to split the string.
When set to None (the default value), will split on any whitespace
character (including \\n \\r \\t \\f and spaces) and will discard
empty strings from the result.
maxsplit
Maximum number of splits (starting from the left).
-1 (the default value) means no limit.
Note, str.split() is mainly useful for data that has been intentionally
delimited. With natural text that includes punctuation, consider using
the regular expression module.
splitlines(self, /, keepends=False) Return a list of the lines in the string, breaking at line boundaries. Line breaks are not included in the resulting list unless keepends is given and true.
startswith(...) S.startswith(prefix[, start[, end]]) -> bool Return True if S starts with the specified prefix, False otherwise. With optional start, test S beginning at that position. With optional end, stop comparing S at that position. prefix can also be a tuple of strings to try.
strip(self, chars=None, /) Return a copy of the string with leading and trailing whitespace removed. If chars is given and not None, remove characters in chars instead.
swapcase(self, /) Convert uppercase characters to lowercase and lowercase characters to uppercase.
title(self, /) Return a version of the string where each word is titlecased. More specifically, words start with uppercased characters and all remaining cased characters have lower case.
translate(self, table, /)
Replace each character in the string using the given translation table.
table
Translation table, which must be a mapping of Unicode ordinals to
Unicode ordinals, strings, or None.
The table must implement lookup/indexing via __getitem__, for instance a
dictionary or list. If this operation raises LookupError, the character is
left untouched. Characters mapped to None are deleted.
upper(self, /) Return a copy of the string converted to uppercase.
zfill(self, width, /) Pad a numeric string with zeros on the left, to fill a field of the given width. The string is never truncated.
decode(self, input, errors='strict')
encode(self, input, errors='strict')
Encodes the object input and returns a tuple (output
object, length consumed).
errors defines the error handling to apply. It defaults to
'strict' handling.
The method may not store state in the Codec instance. Use
StreamWriter for codecs which have to keep state in order to
make encoding efficient.
The encoder must be able to handle zero length input and
return an empty object of the output object type in this
situation.
read(self, size=-1, chars=-1, firstline=False)
Decodes data from the stream self.stream and returns the
resulting object.
chars indicates the number of decoded code points or bytes to
return. read() will never return more data than requested,
but it might return less, if there is not enough available.
size indicates the approximate maximum number of decoded
bytes or code points to read for decoding. The decoder
can modify this setting as appropriate. The default value
-1 indicates to read and decode as much as possible. size
is intended to prevent having to decode huge files in one
step.
If firstline is true, and a UnicodeDecodeError happens
after the first line terminator in the input only the first line
will be returned, the rest of the input will be kept until the
next call to read().
The method should use a greedy read strategy, meaning that
it should read as much data as is allowed within the
definition of the encoding and the given size, e.g. if
optional encoding endings or state markers are available
on the stream, these should be read too.
readline(self, size=None, keepends=True)
Read one line from the input stream and return the
decoded data.
size, if given, is passed as size argument to the
read() method.
readlines(self, sizehint=None, keepends=True)
Read all lines available on the input stream
and return them as a list.
Line breaks are implemented using the codec's decoder
method and are included in the list entries.
sizehint, if given, is ignored since there is no efficient
way to finding the true end-of-line.
reset(self)
Resets the codec buffers used for keeping internal state.
Note that no stream repositioning should take place.
This method is primarily intended to be able to recover
from decoding errors.
seek(self, offset, whence=0)
Set the input stream's current position.
Resets the codec buffers used for keeping state.
StreamReaderWriter instances allow wrapping streams which
work in both read and write modes.
The design is such that one can use the factory functions
returned by the codec.lookup() function to construct the
instance.
read(self, size=-1)
readline(self, size=None)
readlines(self, sizehint=None)
reset(self)
seek(self, offset, whence=0)
write(self, data)
writelines(self, list)
encoding = 'unknown'
StreamRecoder instances translate data from one encoding to another.
They use the complete set of APIs returned by the
codecs.lookup() function to implement their task.
Data written to the StreamRecoder is first decoded into an
intermediate format (depending on the "decode" codec) and then
written to the underlying stream using an instance of the provided
Writer class.
In the other direction, data is read from the underlying stream using
a Reader instance and then encoded and returned to the caller.
read(self, size=-1)
readline(self, size=None)
readlines(self, sizehint=None)
reset(self)
seek(self, offset, whence=0)
write(self, data)
writelines(self, list)
data_encoding = 'unknown'
file_encoding = 'unknown'
decode(self, input, errors='strict')
Decodes the object input and returns a tuple (output
object, length consumed).
input must be an object which provides the bf_getreadbuf
buffer slot. Python strings, buffer objects and memory
mapped files are examples of objects providing this slot.
errors defines the error handling to apply. It defaults to
'strict' handling.
The method may not store state in the Codec instance. Use
StreamReader for codecs which have to keep state in order to
make decoding efficient.
The decoder must be able to handle zero length input and
return an empty object of the output object type in this
situation.
encode(self, input, errors='strict')
Encodes the object input and returns a tuple (output
object, length consumed).
errors defines the error handling to apply. It defaults to
'strict' handling.
The method may not store state in the Codec instance. Use
StreamWriter for codecs which have to keep state in order to
make encoding efficient.
The encoder must be able to handle zero length input and
return an empty object of the output object type in this
situation.
reset(self)
Resets the codec buffers used for keeping internal state.
Calling this method should ensure that the data on the
output is put into a clean state, that allows appending
of new fresh data without having to rescan the whole
stream to recover state.
seek(self, offset, whence=0)
write(self, object)
Writes the object's contents encoded to self.stream.
writelines(self, list)
Writes the concatenated list of strings to the stream
using .write().
EncodedFile(file, data_encoding, file_encoding=None, errors='strict')
Return a wrapped version of file which provides transparent
encoding translation.
Data written to the wrapped file is decoded according
to the given data_encoding and then encoded to the underlying
file using file_encoding. The intermediate data type
will usually be Unicode but depends on the specified codecs.
Bytes read from the file are decoded using file_encoding and then
passed back to the caller encoded using data_encoding.
If file_encoding is not given, it defaults to data_encoding.
errors may be given to define the error handling. It defaults
to 'strict' which causes ValueErrors to be raised in case an
encoding error occurs.
The returned wrapped file object provides two extra attributes
.data_encoding and .file_encoding which reflect the given
parameters of the same name. The attributes can be used for
introspection by Python programs.
ascii_decode(data, errors=None, /)
ascii_encode(str, errors=None, /)
backslashreplace_errors(...) Implements the 'backslashreplace' error handling, which replaces malformed data with a backslashed escape sequence.
charmap_build(map, /)
charmap_decode(data, errors=None, mapping=None, /)
charmap_encode(str, errors=None, mapping=None, /)
decode(obj, encoding='utf-8', errors='strict') Decodes obj using the codec registered for encoding. Default encoding is 'utf-8'. errors may be given to set a different error handling scheme. Default is 'strict' meaning that encoding errors raise a ValueError. Other possible values are 'ignore', 'replace' and 'backslashreplace' as well as any other name registered with codecs.register_error that can handle ValueErrors.
encode(obj, encoding='utf-8', errors='strict') Encodes obj using the codec registered for encoding. The default encoding is 'utf-8'. errors may be given to set a different error handling scheme. Default is 'strict' meaning that encoding errors raise a ValueError. Other possible values are 'ignore', 'replace' and 'backslashreplace' as well as any other name registered with codecs.register_error that can handle ValueErrors.
escape_decode(data, errors=None, /)
escape_encode(data, errors=None, /)
getdecoder(encoding)
Lookup up the codec for the given encoding and return
its decoder function.
Raises a LookupError in case the encoding cannot be found.
getencoder(encoding)
Lookup up the codec for the given encoding and return
its encoder function.
Raises a LookupError in case the encoding cannot be found.
getincrementaldecoder(encoding)
Lookup up the codec for the given encoding and return
its IncrementalDecoder class or factory function.
Raises a LookupError in case the encoding cannot be found
or the codecs doesn't provide an incremental decoder.
getincrementalencoder(encoding)
Lookup up the codec for the given encoding and return
its IncrementalEncoder class or factory function.
Raises a LookupError in case the encoding cannot be found
or the codecs doesn't provide an incremental encoder.
getreader(encoding)
Lookup up the codec for the given encoding and return
its StreamReader class or factory function.
Raises a LookupError in case the encoding cannot be found.
getwriter(encoding)
Lookup up the codec for the given encoding and return
its StreamWriter class or factory function.
Raises a LookupError in case the encoding cannot be found.
ignore_errors(...) Implements the 'ignore' error handling, which ignores malformed data and continues.
iterdecode(iterator, encoding, errors='strict', **kwargs)
Decoding iterator.
Decodes the input strings from the iterator using an IncrementalDecoder.
errors and kwargs are passed through to the IncrementalDecoder
constructor.
iterencode(iterator, encoding, errors='strict', **kwargs)
Encoding iterator.
Encodes the input strings from the iterator using an IncrementalEncoder.
errors and kwargs are passed through to the IncrementalEncoder
constructor.
latin_1_decode(data, errors=None, /)
latin_1_encode(str, errors=None, /)
lookup(encoding, /) Looks up a codec tuple in the Python codec registry and returns a CodecInfo object.
lookup_error(name, /) lookup_error(errors) -> handler Return the error handler for the specified error handling name or raise a LookupError, if no handler exists under this name.
make_encoding_map(decoding_map)
Creates an encoding map from a decoding map.
If a target mapping in the decoding map occurs multiple
times, then that target is mapped to None (undefined mapping),
causing an exception when encountered by the charmap codec
during translation.
One example where this happens is cp875.py which decodes
multiple character to \u001a.
make_identity_dict(rng)
make_identity_dict(rng) -> dict
Return a dictionary where elements of the rng sequence are
mapped to themselves.
namereplace_errors(...)
Implements the 'namereplace' error handling, which replaces an unencodable character with a \N{...} escape sequence.
open(filename, mode='r', encoding=None, errors='strict', buffering=-1)
Open an encoded file using the given mode and return
a wrapped version providing transparent encoding/decoding.
Note: The wrapped version will only accept the object format
defined by the codecs, i.e. Unicode objects for most builtin
codecs. Output is also codec dependent and will usually be
Unicode as well.
Underlying encoded files are always opened in binary mode.
The default file mode is 'r', meaning to open the file in read mode.
encoding specifies the encoding which is to be used for the
file.
errors may be given to define the error handling. It defaults
to 'strict' which causes ValueErrors to be raised in case an
encoding error occurs.
buffering has the same meaning as for the builtin open() API.
It defaults to -1 which means that the default buffer size will
be used.
The returned wrapped file object provides an extra attribute
.encoding which allows querying the used encoding. This
attribute is only available if an encoding was specified as
parameter.
raw_unicode_escape_decode(data, errors=None, final=True, /)
raw_unicode_escape_encode(str, errors=None, /)
readbuffer_encode(data, errors=None, /)
register(search_function, /) Register a codec search function. Search functions are expected to take one argument, the encoding name in all lower case letters, and either return None, or a tuple of functions (encoder, decoder, stream_reader, stream_writer) (or a CodecInfo object).
register_error(errors, handler, /) Register the specified error handler under the name errors. handler must be a callable object, that will be called with an exception instance containing information about the location of the encoding/decoding error and must return a (replacement, new position) tuple.
replace_errors(...) Implements the 'replace' error handling, which replaces malformed data with a replacement marker.
strict_errors(...) Implements the 'strict' error handling, which raises a UnicodeError on coding errors.
unicode_escape_decode(data, errors=None, final=True, /)
unicode_escape_encode(str, errors=None, /)
unregister(search_function, /) Unregister a codec search function and clear the registry's cache. If the search function is not registered, do nothing.
utf_16_be_decode(data, errors=None, final=False, /)
utf_16_be_encode(str, errors=None, /)
utf_16_decode(data, errors=None, final=False, /)
utf_16_encode(str, errors=None, byteorder=0, /)
utf_16_ex_decode(data, errors=None, byteorder=0, final=False, /)
utf_16_le_decode(data, errors=None, final=False, /)
utf_16_le_encode(str, errors=None, /)
utf_32_be_decode(data, errors=None, final=False, /)
utf_32_be_encode(str, errors=None, /)
utf_32_decode(data, errors=None, final=False, /)
utf_32_encode(str, errors=None, byteorder=0, /)
utf_32_ex_decode(data, errors=None, byteorder=0, final=False, /)
utf_32_le_decode(data, errors=None, final=False, /)
utf_32_le_encode(str, errors=None, /)
utf_7_decode(data, errors=None, final=False, /)
utf_7_encode(str, errors=None, /)
utf_8_decode(data, errors=None, final=False, /)
utf_8_encode(str, errors=None, /)
xmlcharrefreplace_errors(...) Implements the 'xmlcharrefreplace' error handling, which replaces an unencodable character with the appropriate XML character reference.
BOM = b'\xff\xfe'
BOM32_BE = b'\xfe\xff'
BOM32_LE = b'\xff\xfe'
BOM64_BE = b'\x00\x00\xfe\xff'
BOM64_LE = b'\xff\xfe\x00\x00'
BOM_BE = b'\xfe\xff'
BOM_LE = b'\xff\xfe'
BOM_UTF16 = b'\xff\xfe'
BOM_UTF16_BE = b'\xfe\xff'
BOM_UTF16_LE = b'\xff\xfe'
BOM_UTF32 = b'\xff\xfe\x00\x00'
BOM_UTF32_BE = b'\x00\x00\xfe\xff'
BOM_UTF32_LE = b'\xff\xfe\x00\x00'
BOM_UTF8 = b'\xef\xbb\xbf'