fsspec

Access LucidLink filespaces from Python data libraries through fsspec.

Install with pip install "lucidlink[fsspec]". fsspec then opens lucidlink://workspace/filespace/path URLs with no import of lucidlink, and so do the libraries that open files through fsspec, such as pandas.

The service account token is the token option: pass token=... when you open a URL with fsspec, or storage_options={"token": ...} to pandas, which hands those options to fsspec. To keep the token out of the code, set the FSSPEC_LUCIDLINK_TOKEN environment variable instead.

PyArrow does not open lucidlink:// URLs: give it a LucidLinkFileSystem as filesystem, as in the example below.

Features

Read, write, list and search files with lucidlink:// URLs
Streaming reads: data is fetched as you read, so part of a large file is read without downloading all of it
Move and rename within a filespace without copying any data
Copy within a filespace and across filespaces
Several filespaces of one workspace through one instance
Sync to LucidLink on demand with sync_all(), or on LucidLinkFileSystem.close()

Usage with pandas

import pandas as pd
df = pd.read_csv(    "lucidlink://workspace/filespace/data.csv",    storage_options={"token": "sa_live:..."},)

Usage with PyArrow

import pyarrow.parquet as pqfrom lucidlink.fsspec import LucidLinkFileSystem
with LucidLinkFileSystem(token="sa_live:...") as fs:    table = pq.read_table(        "lucidlink://workspace/filespace/data.parquet", filesystem=fs    )

Direct usage

from lucidlink.fsspec import LucidLinkFileSystem
with LucidLinkFileSystem(token="sa_live:...") as fs:    with fs.open("lucidlink://workspace/filespace/file.txt", "r") as f:        content = f.read()

class

lucidlink.fsspec.LucidLinkFileSystem

LucidLinkFileSystem(*, token: str | None = None, sync_barrier: SyncBarrier | str = SyncBarrier.ON_UNLINK, auto_mkdir: bool = False, skip_instance_cache: bool = False, **kwargs: Any)

Parameter	Description
`token`	Service account token ("sa_live:...") of the workspace to access. A pickled instance does not carry it; supply it again in the receiving process, e.g. through fsspec's configuration.
`sync_barrier`	What close() does with changes not yet on LucidLink. "on_unlink" (the default) waits until they are uploaded; "none" does not wait, and they are lost. A SyncBarrier works too.
`auto_mkdir`	Whether a method of this instance that writes or moves into a missing directory creates it (LucidLinkFileSystem.open(), pipe_file, mv, ...). True creates the missing directories inside the filespace. False (the default) raises FsNotFoundError instead. put and copy always create them.
`skip_instance_cache`	Passed to fsspec. True creates a new instance, and so a new client, instead of reusing a cached one.
`**kwargs`	Ignored, with a LucidLinkWarning unless it is asynchronous or loop, which some libraries pass.

token

Description: Service account token ("sa_live:...") of the workspace to access. A pickled instance does not carry it; supply it again in the receiving process, e.g. through fsspec's configuration.

sync_barrier

Description: What close() does with changes not yet on LucidLink. "on_unlink" (the default) waits until they are uploaded; "none" does not wait, and they are lost. A SyncBarrier works too.

auto_mkdir

Description: Whether a method of this instance that writes or moves into a missing directory creates it (LucidLinkFileSystem.open(), pipe_file, mv, ...). True creates the missing directories inside the filespace. False (the default) raises FsNotFoundError instead. put and copy always create them.

skip_instance_cache

Description: Passed to fsspec. True creates a new instance, and so a new client, instead of reusing a cached one.

**kwargs

Description: Ignored, with a LucidLinkWarning unless it is asynchronous or loop, which some libraries pass.

fsspec file system for LucidLink filespaces.

Libraries that work with fsspec, such as pandas, can use it to read and write lucidlink://workspace/filespace/path URLs. PyArrow does not open these URLs: give it an instance as filesystem.

Writes upload to LucidLink in the background. They are on LucidLink once sync_all() returns, or LucidLinkFileSystem.close() with sync_barrier="on_unlink". An instance still open at interpreter exit is closed then.

An instance is a context manager: a with block closes it on exit. After LucidLinkFileSystem.close(), every other call raises ClientClosedError; create a new instance for further work.

Each instance has its own Client and is not thread safe: do not use one instance from several threads at the same time.

Clients are only eventually consistent: a client sees changes made through another only after a delay. A file another client has just closed can be missing, empty or truncated for a few seconds, and two clients creating the same new directory conflict: one of the directories ends up as <name>_conflicted. fsspec creates a separate instance, and so a separate client, for each thread, process and set of storage options. With skip_instance_cache=True, it always creates a new one.

Worker processes started with the fork start method, the default on Linux before Python 3.14, cannot use LucidLink while a client runs in the parent: every call raises ClientError. Use the spawn or forkserver start method instead.

Symbolic links are followed in the directories of a path, but never as its last component: methods act on the link itself, and LucidLinkFileSystem.open() on a link raises FsError with errno.ELOOP. A relative link target is resolved from the root of the filespace, not from the link's directory, so use absolute targets.

Errors common to every call that takes a path are not repeated in each Raises section. The first use of a filespace can raise AuthenticationError, AccessDeniedError or RateLimitedError. A URL naming another workspace raises AccessDeniedError, and an unknown filespace FsNotFoundError with the URL as its filename. A malformed URL, or an instance without a token, raises InvalidArgumentError. Beyond these, path errors are those of FileSystem, and an internal error of the filespace raises FilespaceInternalError.

Raises

Exception	Condition
`InvalidArgumentError`	If storage_mode, root_path or sandboxed is given, sync_barrier is invalid, or token is malformed (a ValueError)
`TypeError`	If auto_mkdir is not a bool
`AuthenticationError`	If the token is invalid, expired or revoked
`AccessDeniedError`	If the service account is not allowed into its workspace
`RateLimitedError`	If LucidLink services report too many requests

InvalidArgumentError

Condition: If storage_mode, root_path or sandboxed is given, sync_barrier is invalid, or token is malformed (a ValueError)

TypeError

Condition: If auto_mkdir is not a bool

AuthenticationError

Condition: If the token is invalid, expired or revoked

AccessDeniedError

Condition: If the service account is not allowed into its workspace

RateLimitedError

Condition: If LucidLink services report too many requests

property

transaction: `NoReturn`

Not supported.

Raises

Exception	Condition
`NotImplementedError`	Always

NotImplementedError

Condition: Always

method

cat(path: `str | list[str]`, recursive: `bool` = False, on_error: `Literal['raise', 'omit', 'return']` = 'raise', **kwargs: `Any`) -> `bytes | dict[str, bytes | Exception]`

Fetch the contents of one or more files.

Parameters

Parameter	Description
`path`	LucidLink URL, glob, or list of them
`recursive`	Also fetch the files in the directories that path names
`on_error`	For several files: "raise" raises an error, "omit" leaves that file out, "return" puts the error in the result
`**kwargs`	Passed to cat_file()

path

Description: LucidLink URL, glob, or list of them

recursive

Description: Also fetch the files in the directories that path names

on_error

Description: For several files: "raise" raises an error, "omit" leaves that file out, "return" puts the error in the result

**kwargs

Description: Passed to cat_file()

Returns

The bytes of a single file, or a dict of path to bytes (or error) for several

method

cat_ranges(paths: `list[str]`, starts: `int | None | list[int | None]`, ends: `int | None | list[int | None]`, max_gap: `int | None` = None, on_error: `Literal['return', 'raise']` = 'return', **kwargs: `Any`) -> `list[bytes | Exception]`

Get the contents of byte ranges from one or more files.

Parameters

Parameter	Description
`paths`	LucidLink URLs
`starts`	Start offset of the ranges, one for all paths or one per path; None reads from the start
`ends`	End offset of the ranges, one for all paths or one per path; None reads to the end
`max_gap`	Not supported: anything other than None raises NotImplementedError
`on_error`	"return" puts the error of a range in the result instead of raising it; "raise" raises it
`**kwargs`	Passed to cat_file()

paths

Description: LucidLink URLs

starts

Description: Start offset of the ranges, one for all paths or one per path; None reads from the start

ends

Description: End offset of the ranges, one for all paths or one per path; None reads to the end

max_gap

Description: Not supported: anything other than None raises NotImplementedError

on_error

Description: "return" puts the error of a range in the result instead of raising it; "raise" raises it

**kwargs

Description: Passed to cat_file()

Returns

The bytes of each range, or its error with on_error="return"

method

close() -> `None`

Close the files still open, then the connections, and release resources.

Every step runs even when an earlier one fails: closing the files still open, the final sync (with sync_barrier="on_unlink"), unlinking every filespace and closing the client. Later calls do nothing.

Raises

Exception	Condition
`LucidLinkErrorGroup`	If closing an open file (its data may not have been saved), the final sync, unlinking a filespace or closing the client failed. The instance is closed regardless

LucidLinkErrorGroup

Condition: If closing an open file (its data may not have been saved), the final sync, unlinking a filespace or closing the client failed. The instance is closed regardless

method

cp_file(path1: `str`, path2: `str`, **kwargs: `Any`) -> `None`

Copy one file, recreate a symbolic link, or, for a directory, create path2 without copying its contents.

Works within a filespace and across filespaces; the target's parent directories are created as needed. A symbolic link keeps its link text.

Parameters

Parameter	Description
`path1`	Source LucidLink URL
`path2`	Destination LucidLink URL
`**kwargs`	Ignored

path1

Description: Source LucidLink URL

path2

Description: Destination LucidLink URL

**kwargs

Description: Ignored

Raises

Exception	Condition
`FsNotFoundError`	If path1 does not exist (a FileNotFoundError)
`FsExistsError`	If path1 is a symbolic link and path2 exists (a FileExistsError)
`FsError`	If path2 is a symbolic link (errno.ELOOP)
`shutil.SameFileError`	If both paths name the same file, e.g. spellings that differ only in case or hard links to one file

FsNotFoundError

Condition: If path1 does not exist (a FileNotFoundError)

FsExistsError

Condition: If path1 is a symbolic link and path2 exists (a FileExistsError)

FsError

Condition: If path2 is a symbolic link (errno.ELOOP)

shutil.SameFileError

Condition: If both paths name the same file, e.g. spellings that differ only in case or hard links to one file

method

created(path: `str`) -> `datetime | None`

Return the creation time of a file or directory.

Parameters

Parameter	Description
`path`	LucidLink URL

path

Description: LucidLink URL

Returns

A timezone-aware UTC datetime, or None when the time is unknown (e.g. for a filespace root)

Raises

Exception	Condition
`FsNotFoundError`	If the path does not exist

FsNotFoundError

Condition: If the path does not exist

method

exists(path: `str`, **kwargs: `Any`) -> `bool`

Check if path exists; False when it does not or is a symbolic link.

Errors other than a missing path or a loop of symbolic links are raised, so a failure never reads as "missing". lexists() is True for a link too.

Parameters

Parameter	Description
`path`	LucidLink URL
`**kwargs`	Ignored

path

Description: LucidLink URL

**kwargs

Description: Ignored

method

info(path: `str`, **kwargs: `Any`) -> `dict[str, Any]`

Get information about a file, directory or symbolic link.

Parameters

Parameter	Description
`path`	LucidLink URL
`**kwargs`	Ignored

path

Description: LucidLink URL

**kwargs

Description: Ignored

Returns

The info dict ls() returns per entry with detail. "name" is the full path without the lucidlink:// prefix, "type" is "directory", "link" or "file", "islink" is True for a link, and "mtime" / "created" are in seconds (None when unknown, e.g. for a filespace root)

Raises

Exception	Condition
`FsNotFoundError`	If the path does not exist
`FsError`	If a loop of symbolic links is in the way (errno.ELOOP)

FsNotFoundError

Condition: If the path does not exist

FsError

Condition: If a loop of symbolic links is in the way (errno.ELOOP)

method

isdir(path: `str`) -> `bool`

Check if path is a directory; False when it is not, is a symbolic link, or does not exist.

Any other error is raised, as by exists().

Parameters

Parameter	Description
`path`	LucidLink URL

path

Description: LucidLink URL

method

isfile(path: `str`) -> `bool`

Check if path is a file; False when it is not, is a symbolic link, or does not exist.

Any other error is raised, as by exists().

Parameters

Parameter	Description
`path`	LucidLink URL

path

Description: LucidLink URL

method

lexists(path: `str`, **kwargs: `Any`) -> `bool`

Check if an entry exists at path; True for a symbolic link too.

False when the path does not exist; any other error is raised, as by exists().

Parameters

Parameter	Description
`path`	LucidLink URL
`**kwargs`	Ignored

path

Description: LucidLink URL

**kwargs

Description: Ignored

method

ls(path: `str`, detail: `bool` = True, **kwargs: `Any`) -> `list[str] | list[dict[str, Any]]`

List a directory; a file or a symbolic link lists as itself.

Parameters

Parameter	Description
`path`	LucidLink URL
`detail`	True returns an info dict per entry, False only the paths
`**kwargs`	Ignored

path

Description: LucidLink URL

detail

Description: True returns an info dict per entry, False only the paths

**kwargs

Description: Ignored

Returns

Full paths without the lucidlink:// prefix, or with detail an info dict per entry with "name", "size", "type" ("directory", "link" or "file"), "islink", and "mtime" / "created" in seconds (None when unknown)

Raises

Exception	Condition
`FsNotFoundError`	If the path does not exist

FsNotFoundError

Condition: If the path does not exist

method

makedirs(path: `str`, exist_ok: `bool` = False) -> `None`

Create a directory and any missing parent directories.

A workspace or filespace is never created.

Parameters

Parameter	Description
`path`	LucidLink URL
`exist_ok`	Accept an existing directory at path, a filespace root included

path

Description: LucidLink URL

exist_ok

Description: Accept an existing directory at path, a filespace root included

Raises

Exception	Condition
`FsExistsError`	If path exists and exist_ok is false, or exists and is not a directory (a FileExistsError)
`FsNotADirectoryError`	If a parent component is not a directory (a NotADirectoryError)

FsExistsError

Condition: If path exists and exist_ok is false, or exists and is not a directory (a FileExistsError)

FsNotADirectoryError

Condition: If a parent component is not a directory (a NotADirectoryError)

method

mkdir(path: `str`, create_parents: `bool` = True, **kwargs: `Any`) -> `None`

Create a directory.

Parameters

Parameter	Description
`path`	LucidLink URL
`create_parents`	Also create missing parent directories
`**kwargs`	Ignored

path

Description: LucidLink URL

create_parents

Description: Also create missing parent directories

**kwargs

Description: Ignored

Raises

Exception	Condition
`FsExistsError`	If path already exists, a filespace root included (a FileExistsError)
`FsNotFoundError`	If the parent directory does not exist and create_parents is false (a FileNotFoundError)
`FsNotADirectoryError`	If a parent component is not a directory (a NotADirectoryError)

FsExistsError

Condition: If path already exists, a filespace root included (a FileExistsError)

FsNotFoundError

Condition: If the parent directory does not exist and create_parents is false (a FileNotFoundError)

FsNotADirectoryError

Condition: If a parent component is not a directory (a NotADirectoryError)

method

modified(path: `str`) -> `datetime | None`

Return the last modification time of a file or directory.

Parameters

Parameter	Description
`path`	LucidLink URL

path

Description: LucidLink URL

Returns

A timezone-aware UTC datetime, or None when the time is unknown (e.g. for a filespace root)

Raises

Exception	Condition
`FsNotFoundError`	If the path does not exist

FsNotFoundError

Condition: If the path does not exist

method

mv(path1: `str`, path2: `str`, recursive: `bool` = False, maxdepth: `int | None` = None, **kwargs: `Any`) -> `None`

Move or rename a file or directory.

path1 moves to path2, or into it when path2 is an existing directory. An existing file at path2 is replaced by a file, but not atomically: another client can briefly find no file there. Moving into a directory never replaces anything. A directory always moves whole. Within a filespace a move copies no data; across filespaces the data is copied, then deleted, and symbolic links are recreated.

Each path is one entry, taken literally: globs and lists are not accepted. To move several entries, call mv() once per path, e.g. for each path from glob().

Parameters

Parameter	Description
`path1`	Source LucidLink URL
`path2`	Destination LucidLink URL
`recursive`	Ignored: a directory always moves whole
`maxdepth`	Not supported
`**kwargs`	Ignored

path1

Description: Source LucidLink URL

path2

Description: Destination LucidLink URL

recursive

Description: Ignored: a directory always moves whole

maxdepth

Description: Not supported

**kwargs

Description: Ignored

Raises

Exception	Condition
`FsNotFoundError`	If the source does not exist, or the destination's parent does not exist and auto_mkdir is off (a FileNotFoundError)
`FsExistsError`	If path2 exists and it or the source is not a file, or path2 is a directory that already holds an entry with the source's name (a FileExistsError)
`FsNotADirectoryError`	If the destination's parent is a file (a NotADirectoryError)
`FsError`	If the source is a filespace root, or a directory moved into itself (errno.EINVAL)
`NotImplementedError`	If maxdepth is given
`TypeError`	If either path is not a single path (a list, for example)

FsNotFoundError

Condition: If the source does not exist, or the destination's parent does not exist and auto_mkdir is off (a FileNotFoundError)

FsExistsError

Condition: If path2 exists and it or the source is not a file, or path2 is a directory that already holds an entry with the source's name (a FileExistsError)

FsNotADirectoryError

Condition: If the destination's parent is a file (a NotADirectoryError)

FsError

Condition: If the source is a filespace root, or a directory moved into itself (errno.EINVAL)

NotImplementedError

Condition: If maxdepth is given

TypeError

Condition: If either path is not a single path (a list, for example)

method

open(path: `str`, mode: `str` = 'rb', block_size: `int | None` = None, cache_options: `dict[str, Any] | None` = None, compression: `str | None` = None, **kwargs: `Any`) -> `Any`

Open a file and return a file object.

Parameters

Parameter	Description
`path`	LucidLink URL
`mode`	How to open the file, "rb" by default, as for FileSystem.open(): "r", "w" or "a", a "+" to add reading or writing, and "b" for binary, otherwise text. Exclusive creation ("x") is not supported.
`block_size`	Buffer size; None uses the default
`cache_options`	Ignored
`compression`	Compression to apply, e.g. "gzip", or "infer" from the file name
`**kwargs`	encoding, errors and newline for text modes

path

Description: LucidLink URL

mode

Description: How to open the file, "rb" by default, as for FileSystem.open(): "r", "w" or "a", a "+" to add reading or writing, and "b" for binary, otherwise text. Exclusive creation ("x") is not supported.

block_size

Description: Buffer size; None uses the default

cache_options

Description: Ignored

compression

Description: Compression to apply, e.g. "gzip", or "infer" from the file name

**kwargs

Description: encoding, errors and newline for text modes

Returns

A text file object for a text mode, otherwise a binary one

Raises

Exception	Condition
`FsNotFoundError`	If the file does not exist (read modes), or its parent directory does not exist in a mode that creates the file and auto_mkdir is false (a FileNotFoundError)
`FsIsADirectoryError`	If the path is a directory (an IsADirectoryError)
`FsError`	If the path is a symbolic link (errno.ELOOP)
`InvalidArgumentError`	If mode is invalid (a ValueError)

FsNotFoundError

Condition: If the file does not exist (read modes), or its parent directory does not exist in a mode that creates the file and auto_mkdir is false (a FileNotFoundError)

FsIsADirectoryError

Condition: If the path is a directory (an IsADirectoryError)

FsError

Condition: If the path is a symbolic link (errno.ELOOP)

InvalidArgumentError

Condition: If mode is invalid (a ValueError)

method

rename(path1: `str`, path2: `str`, **kwargs: `Any`) -> `None`

Rename a file or directory; the same as mv().

Parameters

Parameter	Description
`path1`	Source LucidLink URL
`path2`	Destination LucidLink URL
`**kwargs`	Passed to mv()

path1

Description: Source LucidLink URL

path2

Description: Destination LucidLink URL

**kwargs

Description: Passed to mv()

method

rm(path: `str | list[str]`, recursive: `bool` = False, maxdepth: `int | None` = None) -> `None`

Remove files, directories or symbolic links.

Without recursive, a directory is removed only when it is empty.

Parameters

Parameter	Description
`path`	LucidLink URL, glob, or list of them
`recursive`	Also remove directories with their contents
`maxdepth`	Maximum depth to descend when recursive is set

path

Description: LucidLink URL, glob, or list of them

recursive

Description: Also remove directories with their contents

maxdepth

Description: Maximum depth to descend when recursive is set

Raises

Exception	Condition
`FsNotFoundError`	If the path does not exist (a FileNotFoundError)
`FsDirectoryNotEmptyError`	If a directory is not empty and recursive is not set (an OSError)

FsNotFoundError

Condition: If the path does not exist (a FileNotFoundError)

FsDirectoryNotEmptyError

Condition: If a directory is not empty and recursive is not set (an OSError)

method

rm_file(path: `str`) -> `None`

Delete one file, one symbolic link, or one empty directory.

Parameters

Parameter	Description
`path`	LucidLink URL

path

Description: LucidLink URL

Raises

Exception	Condition
`FsNotFoundError`	If the path does not exist (a FileNotFoundError)
`FsDirectoryNotEmptyError`	If the path is a directory that is not empty (an OSError)

FsNotFoundError

Condition: If the path does not exist (a FileNotFoundError)

FsDirectoryNotEmptyError

Condition: If the path is a directory that is not empty (an OSError)

method

rmdir(path: `str`) -> `None`

Remove an empty directory.

Parameters

Parameter	Description
`path`	LucidLink URL

path

Description: LucidLink URL

Raises

Exception	Condition
`FsNotFoundError`	If the path does not exist (a FileNotFoundError)
`FsNotADirectoryError`	If the path is not a directory (a NotADirectoryError)
`FsDirectoryNotEmptyError`	If the directory is not empty (an OSError)

FsNotFoundError

Condition: If the path does not exist (a FileNotFoundError)

FsNotADirectoryError

Condition: If the path is not a directory (a NotADirectoryError)

FsDirectoryNotEmptyError

Condition: If the directory is not empty (an OSError)

method

start_transaction() -> `NoReturn`

Not supported.

Raises

Exception	Condition
`NotImplementedError`	Always

NotImplementedError

Condition: Always

method

sync_all() -> `None`

Synchronize all pending changes to LucidLink services.

Syncs every filespace this instance holds, even when one fails, and returns once their changes are on LucidLink. Other clients see the changes only after a delay.

Raises

Exception	Condition
`LucidLinkErrorGroup`	If one or more filespaces failed to sync. A FilespaceInternalError member means that filespace's pending changes are lost; its filespace_id names it

LucidLinkErrorGroup

Condition: If one or more filespaces failed to sync. A FilespaceInternalError member means that filespace's pending changes are lost; its filespace_id names it

method

touch(path: `str`, truncate: `bool` = True, **kwargs: `Any`) -> `None`

Create an empty file, or empty an existing one.

Parameters

Parameter	Description
`path`	LucidLink URL
`truncate`	True (default) empties an existing file. False only creates a file that does not exist: the timestamps of an existing file cannot be updated
`**kwargs`	Passed to open()

path

Description: LucidLink URL

truncate

Description: True (default) empties an existing file. False only creates a file that does not exist: the timestamps of an existing file cannot be updated

**kwargs

Description: Passed to open()

Raises

Exception	Condition
`NotImplementedError`	If truncate is False and the file exists
`FsNotFoundError`	If the parent directory does not exist and auto_mkdir is false (a FileNotFoundError)
`FsIsADirectoryError`	If the path is a directory (an IsADirectoryError)
`FsError`	If the path is a symbolic link (errno.ELOOP)

NotImplementedError

Condition: If truncate is False and the file exists

FsNotFoundError

Condition: If the parent directory does not exist and auto_mkdir is false (a FileNotFoundError)

FsIsADirectoryError

Condition: If the path is a directory (an IsADirectoryError)

FsError

Condition: If the path is a symbolic link (errno.ELOOP)

method

walk(path: `str`, maxdepth: `int | None` = None, topdown: `bool` = True, on_error: `Literal['omit', 'raise'] | Callable[[OSError], object] | None` = None, **kwargs: `Any`) -> `Iterator[tuple[str, Any, Any]]`

Walk the tree below path, yielding (root, dirs, files) for each directory.

find(), glob() and du() use it too. A directory that is missing, not a directory or not readable is omitted; any other error is raised, so a partial tree is never returned as if it were complete. A symbolic link as path is listed as itself.

Parameters

Parameter	Description
`path`	LucidLink URL
`maxdepth`	Maximum depth to descend; None for no limit
`topdown`	Yield a directory before its subdirectories
`on_error`	For the listing of path only: "omit" omits an OSError, "raise" raises it, a callable receives it. None omits path when it is missing, not a directory or not readable, and raises any other error. An error that is not an OSError is always raised
`**kwargs`	detail=True yields info dicts instead of names

path

Description: LucidLink URL

maxdepth

Description: Maximum depth to descend; None for no limit

topdown

Description: Yield a directory before its subdirectories

on_error

Description: For the listing of path only: "omit" omits an OSError, "raise" raises it, a callable receives it. None omits path when it is missing, not a directory or not readable, and raises any other error. An error that is not an OSError is always raised

**kwargs

Description: detail=True yields info dicts instead of names

Raises

Exception	Condition
`InvalidArgumentError`	If maxdepth is less than 1

InvalidArgumentError

Condition: If maxdepth is less than 1

fsspec

Features

lucidlink.fsspec.LucidLinkFileSystem

Raises

transaction: NoReturn

Raises

cat(path: str | list[str], recursive: bool = False, on_error: Literal['raise', 'omit', 'return'] = 'raise', **kwargs: Any) -> bytes | dict[str, bytes | Exception]

Parameters

Returns

cat_ranges(paths: list[str], starts: int | None | list[int | None], ends: int | None | list[int | None], max_gap: int | None = None, on_error: Literal['return', 'raise'] = 'return', **kwargs: Any) -> list[bytes | Exception]

Parameters

Returns

close() -> None

Raises

cp_file(path1: str, path2: str, **kwargs: Any) -> None

Parameters

Raises

created(path: str) -> datetime | None

Parameters

Returns

Raises

exists(path: str, **kwargs: Any) -> bool

Parameters

info(path: str, **kwargs: Any) -> dict[str, Any]

Parameters

Returns

Raises

isdir(path: str) -> bool

Parameters

isfile(path: str) -> bool

Parameters

lexists(path: str, **kwargs: Any) -> bool

Parameters

ls(path: str, detail: bool = True, **kwargs: Any) -> list[str] | list[dict[str, Any]]

Parameters

Returns

Raises

makedirs(path: str, exist_ok: bool = False) -> None

Parameters

Raises

mkdir(path: str, create_parents: bool = True, **kwargs: Any) -> None

Parameters

Raises

modified(path: str) -> datetime | None

Parameters

Returns

Raises

mv(path1: str, path2: str, recursive: bool = False, maxdepth: int | None = None, **kwargs: Any) -> None

Parameters

Raises

open(path: str, mode: str = 'rb', block_size: int | None = None, cache_options: dict[str, Any] | None = None, compression: str | None = None, **kwargs: Any) -> Any

Parameters

Returns

Raises

rename(path1: str, path2: str, **kwargs: Any) -> None

Parameters

rm(path: str | list[str], recursive: bool = False, maxdepth: int | None = None) -> None

Parameters

Raises

rm_file(path: str) -> None

Parameters

Raises

rmdir(path: str) -> None

Parameters

Raises

start_transaction() -> NoReturn

Raises

sync_all() -> None

Raises

touch(path: str, truncate: bool = True, **kwargs: Any) -> None

Parameters

Raises

walk(path: str, maxdepth: int | None = None, topdown: bool = True, on_error: Literal['omit', 'raise'] | Callable[[OSError], object] | None = None, **kwargs: Any) -> Iterator[tuple[str, Any, Any]]

Parameters

Raises

transaction: `NoReturn`

cat(path: `str | list[str]`, recursive: `bool` = False, on_error: `Literal['raise', 'omit', 'return']` = 'raise', **kwargs: `Any`) -> `bytes | dict[str, bytes | Exception]`

cat_ranges(paths: `list[str]`, starts: `int | None | list[int | None]`, ends: `int | None | list[int | None]`, max_gap: `int | None` = None, on_error: `Literal['return', 'raise']` = 'return', **kwargs: `Any`) -> `list[bytes | Exception]`

close() -> `None`

cp_file(path1: `str`, path2: `str`, **kwargs: `Any`) -> `None`

created(path: `str`) -> `datetime | None`

exists(path: `str`, **kwargs: `Any`) -> `bool`

info(path: `str`, **kwargs: `Any`) -> `dict[str, Any]`

isdir(path: `str`) -> `bool`

isfile(path: `str`) -> `bool`

lexists(path: `str`, **kwargs: `Any`) -> `bool`

ls(path: `str`, detail: `bool` = True, **kwargs: `Any`) -> `list[str] | list[dict[str, Any]]`

makedirs(path: `str`, exist_ok: `bool` = False) -> `None`

mkdir(path: `str`, create_parents: `bool` = True, **kwargs: `Any`) -> `None`

modified(path: `str`) -> `datetime | None`

mv(path1: `str`, path2: `str`, recursive: `bool` = False, maxdepth: `int | None` = None, **kwargs: `Any`) -> `None`

open(path: `str`, mode: `str` = 'rb', block_size: `int | None` = None, cache_options: `dict[str, Any] | None` = None, compression: `str | None` = None, **kwargs: `Any`) -> `Any`

rename(path1: `str`, path2: `str`, **kwargs: `Any`) -> `None`

rm(path: `str | list[str]`, recursive: `bool` = False, maxdepth: `int | None` = None) -> `None`

rm_file(path: `str`) -> `None`

rmdir(path: `str`) -> `None`

start_transaction() -> `NoReturn`

sync_all() -> `None`

touch(path: `str`, truncate: `bool` = True, **kwargs: `Any`) -> `None`

walk(path: `str`, maxdepth: `int | None` = None, topdown: `bool` = True, on_error: `Literal['omit', 'raise'] | Callable[[OSError], object] | None` = None, **kwargs: `Any`) -> `Iterator[tuple[str, Any, Any]]`