🧿 `safer`: A safer writer 🧿

safer wraps file streams, sockets, or a callable, and offers a drop-in replacement for regular old open().

Quick summary

A tiny example

import safer

with safer.open(filename, 'w') as fp:
    fp.write('one')
    print('two', file=fp)
    raise ValueError
    # filename was not written.

How to use

Use pip to install safer from the command line: pip install safer.

Tested on Python 3.4 - 3.11. An old Python 2.7 version is here.

See the Medium article here

The details

safer helps prevent programmer error from corrupting files, socket connections, or generalized streams by writing a whole file or nothing.

It does not prevent concurrent modification of files from other threads or processes: if you need atomic file writing, see https://pypi.org/project/atomicwrites/

It also has a useful dry_run setting to let you test your code without actually overwriting the target file.

safer.writer() wraps an existing writer, socket or stream and writes a whole response or nothing
safer.open() is a drop-in replacement for built-in open that writes a whole file or nothing
safer.closer() returns a stream like from safer.write() that also closes the underlying stream or callable when it closes.
safer.dump() is like a safer json.dump() which can be used for any serialization protocol, including Yaml and Toml, and also allows you to write to file streams or any other callable.
safer.printer() is safer.open() except that it yields a a function that prints to the stream.

By default, safer buffers the written data in memory in a io.StringIO or io.BytesIO.

For very large files, safer.open() has a temp_file argument which writes the data to a temporary file on disk, which is moved over using os.rename if the operation completes successfully. This functionality does not work on Windows. (In fact, it's unclear if any of this works on Windows, but that certainly won't. Windows developer solicted!)

Example: `safer.writer()`

safer.writer() wraps an existing stream - a writer, socket, or callback - in a temporary stream which is only copied to the target stream at close(), and only if no exception was raised.

Suppose sock = socket.socket(*args).

The old, dangerous way goes like this.

try:
    write_header(sock)
    write_body(sock)   # Exception is thrown here
    write_footer(sock)
 except Exception:
    write_error(sock)  # Oops, the header was already written

With safer you write all or nothing:

try:
    with safer.writer(sock) as s:
        write_header(s)
        write_body(s)  # Exception is thrown here
        write_footer(s)
 except Exception:
    write_error(sock)  # Nothing has been written

Example: `safer.open()` and json

safer.open() is a a drop-in replacement for built-in open() except that when used as a context, it leaves the original file unchanged on failure.

It's easy to write broken JSON if something within it doesn't serialize.

with open(filename, 'w') as fp:
    json.dump(data, fp)
    # If an exception is raised, the file is empty or partly written

safer prevents this:

with safer.open(filename, 'w') as fp:
    json.dump(data, fp)
    # If an exception is raised, the file is unchanged.

safer.open(filename) returns a file stream fp like open(filename) would, except that fp writes to memory stream or a temporary file in the same directory.

If fp is used as a context manager and an exception is raised, then the property fp.safer_failed on the stream is automatically set to True.

And when fp.close() is called, the cached data is stored in filename - unless fp.safer_failed is true.

Example: `safer.printer()`

safer.printer() is similar to safer.open() except it yields a function that prints to the open file - it's very convenient for printing text.

Like safer.open(), if an exception is raised within its context manager, the original file is left unchanged.

Before.

with open(file, 'w') as fp:
    for item in items:
        print(item, file=fp)
    # Prints lines until the first exception

With safer

with safer.printer(file) as print:
    for item in items:
        print(item)
    # Either the whole file is written, or nothing

API Documentation

`closer(stream, is_binary=None, close_on_exit=True, **kwds)`

Like safer.writer() but with close_on_exit=True by default

ARGUMENTS Same as for safer.writer()

Source code in safer/safer.py

def closer(stream, is_binary=None, close_on_exit=True, **kwds):
    """
    Like `safer.writer()` but with `close_on_exit=True` by default

    ARGUMENTS
      Same as for `safer.writer()`
    """
    return writer(stream, is_binary, close_on_exit, **kwds)

`dump(obj, stream=None, dump=None, **kwargs)`

Safely serialize obj as a formatted stream to fp`` (a.write()-supporting file-like object, or a filename), usingjson.dump` by default

ARGUMENTS obj: The object to be serialized

stream: A file stream, a socket, or a callable that will receive data. If stream is None, output is written to sys.stdout. If stream is a string or Path, the file with that name is opened for writing.

dump: A function or module or the name of a function or module to dump data. If None, default to `json.dump``.

kwargs: Additional arguments to dump.

Source code in safer/safer.py

def dump(obj, stream=None, dump=None, **kwargs):
    """
    Safely serialize `obj` as a formatted stream to `fp`` (a
    `.write()`-supporting file-like object, or a filename),
    using `json.dump` by default

    ARGUMENTS
      obj:
        The object to be serialized

      stream:
        A file stream, a socket, or a callable that will receive data.
        If stream is `None`, output is written to `sys.stdout`.
        If stream is a string or `Path`, the file with that name is opened for
        writing.

      dump:
        A function or module or the name of a function or module to dump data.
        If `None`, default to `json.dump``.

      kwargs:
        Additional arguments to `dump`.
    """
    if isinstance(stream, str):
        name = stream
        is_binary = False
    else:
        name = getattr(stream, 'name', None)
        mode = getattr(stream, 'mode', None)
        if name and mode:
            is_binary = 'b' in mode
        else:
            is_binary = hasattr(stream, 'recv') and hasattr(stream, 'send')

    if name and not dump:
        dump = Path(name).suffix[1:] or None
        if dump == 'yml':
            dump = 'yaml'

    if isinstance(dump, str):
        try:
            dump = __import__(dump)
        except ImportError:
            if '.' not in dump:
                raise
            mod, name = dump.rsplit('.', maxsplit=1)
            dump = getattr(__import__(mod), name)

    if dump is None:
        dump = json.dump

    elif not callable(dump):
        try:
            dump = dump.safe_dump
        except AttributeError:
            dump = dump.dump

    with writer(stream) as fp:
        if is_binary:
            write = fp.write
            fp.write = lambda s: write(s.encode('utf-8'))
        return dump(obj, fp)

`open(name, mode='r', buffering=-1, encoding=None, errors=None, newline=None, closefd=True, opener=None, make_parents=False, delete_failures=True, temp_file=False, dry_run=False)`

Parameters:

Name	Type	Description	Default
`make_parents`	`bool`	If true, create the parent directory of the file if needed	`False`
`delete_failures`	`bool`	If false, any temporary files created are not deleted if there is an exception.	`True`
`temp_file`	`bool`	If `temp_file` is truthy, write to a disk file and use os.rename() at the end, otherwise cache the writes in memory. If `temp_file` is a string, use it as the name of the temporary file, otherwise select one in the same directory as the target file, or in the system tempfile for streams that aren't files.	`False`
`dry_run`	`bool`	If dry_run is True, the file is not written to at all	`False`

The remaining arguments are the same as for built-in open().

safer.open() is a drop-in replacement for built-inopen()`. It returns a stream which only overwrites the original file when close() is called, and only if there was no failure.

It works as follows:

If a stream fp return from safer.open() is used as a context manager and an exception is raised, the property fp.safer_failed is set to True.

In the method fp.close(), if fp.safer_failed is not set, then the cached results replace the original file, successfully completing the write.

If fp.safer_failed is true, then if delete_failures is true, the temporary file is deleted.

If the mode argument contains either 'a' (append), or '+' (update), then the original file will be copied to the temporary file before writing starts.

Note that if the temp_file argument is set, safer uses an extra temporary file which is renamed over the file only after the stream closes without failing. This uses as much disk space as the old and new files put together.

Source code in safer/safer.py

def open(
    name: Union[Path, str],
    mode: str = 'r',
    buffering: bool = -1,
    encoding: Optional[str] = None,
    errors: Optional[str] = None,
    newline: Optional[str] = None,
    closefd: bool = True,
    opener: Optional[Callable] = None,
    make_parents: bool = False,
    delete_failures: bool = True,
    temp_file: bool = False,
    dry_run: bool = False,
) -> IO:
    """
    Args:
      make_parents: If true, create the parent directory of the file if needed

      delete_failures: If false, any temporary files created are not deleted
        if there is an exception.

      temp_file: If `temp_file` is truthy, write to a disk file and use
          os.rename() at the end, otherwise cache the writes in memory.

          If `temp_file` is a string, use it as the name of the temporary
          file, otherwise select one in the same directory as the target
          file, or in the system tempfile for streams that aren't files.

      dry_run:
         If dry_run is True, the file is not written to at all

    The remaining arguments are the same as for built-in `open()`.

    `safer.open() is a drop-in replacement for built-in`open()`. It returns a
    stream which only overwrites the original file when close() is called, and
    only if there was no failure.

    It works as follows:

    If a stream `fp` return from `safer.open()` is used as a context
    manager and an exception is raised, the property `fp.safer_failed` is
    set to `True`.

    In the method `fp.close()`, if `fp.safer_failed` is *not* set, then the
    cached results replace the original file, successfully completing the
    write.

    If `fp.safer_failed` is true, then if `delete_failures` is true, the
    temporary file is deleted.

    If the `mode` argument contains either `'a'` (append), or `'+'`
    (update), then the original file will be copied to the temporary file
    before writing starts.

    Note that if the `temp_file` argument is set, `safer` uses an extra
    temporary file which is renamed over the file only after the stream closes
    without failing. This uses as much disk space as the old and new files put
    together.
    """
    is_copy = '+' in mode or 'a' in mode
    is_read = 'r' in mode and not is_copy
    is_binary = 'b' in mode

    kwargs = dict(
        encoding=encoding, errors=errors, newline=newline, opener=opener
    )

    if isinstance(name, Path):
        name = str(name)

    if not isinstance(name, str):
        raise TypeError('`name` must be string, not %s' % type(name).__name__)

    name = os.path.realpath(name)
    parent = os.path.dirname(os.path.abspath(name))
    if not os.path.exists(parent):
        if not make_parents:
            raise IOError('Directory does not exist')
        os.makedirs(parent)

    def simple_open():
        return __builtins__['open'](name, mode, buffering, **kwargs)

    def simple_write(value):
        with simple_open() as fp:
            fp.write(value)

    if is_read:
        return simple_open()

    if not temp_file:
        if '+' in mode:
            raise ValueError('+ mode requires a temp_file argument')

        if callable(dry_run):
            write = dry_run
        else:
            write = len if dry_run else simple_write

        fp = _MemoryStreamCloser(write, True, is_binary).fp
        fp.mode = mode
        return fp

    if not closefd:
        raise ValueError('Cannot use closefd=False with file name')

    if is_binary:
        if 't' in mode:
            raise ValueError('can\'t have text and binary mode at once')
        if newline:
            raise ValueError('binary mode doesn\'t take a newline argument')
        if encoding:
            raise ValueError('binary mode doesn\'t take an encoding argument')
        if errors:
            raise ValueError('binary mode doesn\'t take an errors argument')

    if 'x' in mode and os.path.exists(name):
        raise FileExistsError("File exists: '%s'" % name)

    if buffering == -1:
        buffering = io.DEFAULT_BUFFER_SIZE

    closer = _FileRenameCloser(
        name, temp_file, delete_failures, parent, dry_run
    )

    if is_copy and os.path.exists(name):
        shutil.copy2(name, closer.temp_file)

    return closer._make_stream(buffering, mode, **kwargs)

`printer(name, mode='w', *args, **kwargs)`

A context manager that yields a function that prints to the opened file, only writing to the original file at the exit of the context, and only if there was no exception thrown

ARGUMENTS Same as for safer.open()

Source code in safer/safer.py

@contextlib.contextmanager
def printer(name, mode='w', *args, **kwargs):
    """
    A context manager that yields a function that prints to the opened file,
    only writing to the original file at the exit of the context,
    and only if there was no exception thrown

    ARGUMENTS
      Same as for `safer.open()`
    """
    if 'r' in mode and '+' not in mode:
        raise IOError('File not open for writing')

    if 'b' in mode:
        raise ValueError('Cannot print to a file open in binary mode')

    with open(name, mode, *args, **kwargs) as fp:
        yield functools.partial(print, file=fp)

`writer(stream=None, is_binary=None, close_on_exit=False, temp_file=False, chunk_size=1048576, delete_failures=True, dry_run=False)`

Write safely to file streams, sockets and callables.

safer.writer yields an in-memory stream that you can write to, but which is only written to the original stream if the context finishes without raising an exception.

Because the actual writing happens at the end, it's possible to block indefinitely when the context exits if the underlying socket, stream or callable does!

Parameters:

Name	Type	Description	Default
`stream`	`Union[Callable, None, IO, Path, str]`	A file stream, a socket, or a callable that will receive data. If stream is `None`, output is written to `sys.stdout` If stream is a string or `Path`, the file with that name is opened for writing.	`None`
`is_binary`	`Optional[bool]`	Is `stream` a binary stream? If `is_binary` is `None`, deduce whether it's a binary file from the stream, or assume it's text otherwise.	`None`
`close_on_exit`	`bool`	If True, the underlying stream is closed when the writer closes	`False`
`temp_file`	`bool`	If `temp_file` is truthy, write to a disk file and use os.rename() at the end, otherwise cache the writes in memory. If `temp_file` is a string, use it as the name of the temporary file, otherwise select one in the same directory as the target file, or in the system tempfile for streams that aren't files.	`False`
`chunk_size`	`int`	Chunk size, in bytes for transfer data from the temporary file to the underlying stream.	`1048576`
`delete_failures`	`bool`	If false, any temporary files created are not deleted if there is an exception.	`True`
`dry_run`	`Union[bool, Callable]`	If `dry_run` is truthy, the stream or file is left unchanged. If `dry_run` is also callable, the results of the stream are passed to `dry_run()` rather than being written to the stream.	`False`

Source code in safer/safer.py

def writer(
    stream: Union[Callable, None, IO, Path, str] = None,
    is_binary: Optional[bool] = None,
    close_on_exit: bool = False,
    temp_file: bool = False,
    chunk_size: int = 0x100000,
    delete_failures: bool = True,
    dry_run: Union[bool, Callable] = False,
) -> Union[Callable, IO]:
    """
    Write safely to file streams, sockets and callables.

    `safer.writer` yields an in-memory stream that you can write
    to, but which is only written to the original stream if the
    context finishes without raising an exception.

    Because the actual writing happens at the end, it's possible to block
    indefinitely when the context exits if the underlying socket, stream or
    callable does!

    Args:
      stream: A file stream, a socket, or a callable that will receive data.

          If stream is `None`, output is written to `sys.stdout`

          If stream is a string or `Path`, the file with that name is
          opened for writing.

      is_binary: Is `stream` a binary stream?

          If `is_binary` is ``None``, deduce whether it's a binary file from
          the stream, or assume it's text otherwise.

      close_on_exit: If True, the underlying stream is closed when the writer
        closes

      temp_file: If `temp_file` is truthy, write to a disk file and use
          os.rename() at the end, otherwise cache the writes in memory.

          If `temp_file` is a string, use it as the name of the temporary
          file, otherwise select one in the same directory as the target
          file, or in the system tempfile for streams that aren't files.

      chunk_size: Chunk size, in bytes for transfer data from the temporary
          file to the underlying stream.

      delete_failures: If false, any temporary files created are not deleted
        if there is an exception.

      dry_run: If `dry_run` is truthy, the stream or file is left unchanged.

        If `dry_run` is also callable, the results of the stream are passed to
        `dry_run()` rather than being written to the stream.
    """
    if isinstance(stream, (str, Path)):
        mode = 'wb' if is_binary else 'w'
        return open(
            stream, mode, delete_failures=delete_failures, dry_run=dry_run
        )

    stream = stream or sys.stdout

    if callable(dry_run):
        write, dry_run = dry_run, True
    elif dry_run:
        write = len
    else:
        write = getattr(stream, 'write', None)

    send = getattr(stream, 'send', None)
    mode = getattr(stream, 'mode', None)

    if dry_run:
        close_on_exit = False

    if close_on_exit and stream in (sys.stdout, sys.stderr):
        raise ValueError('You cannot close stdout or stderr')

    if write and mode:
        if not set('w+a').intersection(mode):
            raise ValueError('Stream mode "%s" is not a write mode' % mode)

        binary_mode = 'b' in mode
        if is_binary is not None and is_binary is not binary_mode:
            raise ValueError('is_binary is inconsistent with the file stream')

        is_binary = binary_mode

    elif dry_run:
        pass

    elif send and hasattr(stream, 'recv'):  # It looks like a socket:
        if not (is_binary is None or is_binary is True):
            raise ValueError('is_binary=False is inconsistent with a socket')

        write = send
        is_binary = True

    elif callable(stream):
        write = stream

    else:
        raise ValueError('Stream is not a file, a socket, or callable')

    if temp_file:
        closer = _FileStreamCloser(
            write,
            close_on_exit,
            is_binary,
            temp_file,
            chunk_size,
            delete_failures,
        )
    else:
        closer = _MemoryStreamCloser(write, close_on_exit, is_binary)

    if send is write:
        closer.fp.send = write

    return closer.fp

🧿 safer: A safer writer 🧿

Quick summary

A tiny example

How to use

The details

Example: safer.writer()

Example: safer.open() and json

Example: safer.printer()

API Documentation

closer(stream, is_binary=None, close_on_exit=True, **kwds)

dump(obj, stream=None, dump=None, **kwargs)

open(name, mode='r', buffering=-1, encoding=None, errors=None, newline=None, closefd=True, opener=None, make_parents=False, delete_failures=True, temp_file=False, dry_run=False)

printer(name, mode='w', *args, **kwargs)

writer(stream=None, is_binary=None, close_on_exit=False, temp_file=False, chunk_size=1048576, delete_failures=True, dry_run=False)

About this project

🧿 `safer`: A safer writer 🧿

Example: `safer.writer()`

Example: `safer.open()` and json

Example: `safer.printer()`

`closer(stream, is_binary=None, close_on_exit=True, **kwds)`

`dump(obj, stream=None, dump=None, **kwargs)`

`open(name, mode='r', buffering=-1, encoding=None, errors=None, newline=None, closefd=True, opener=None, make_parents=False, delete_failures=True, temp_file=False, dry_run=False)`

`printer(name, mode='w', *args, **kwargs)`

`writer(stream=None, is_binary=None, close_on_exit=False, temp_file=False, chunk_size=1048576, delete_failures=True, dry_run=False)`