Compare commits

..

36 commits

Author SHA1 Message Date
CismonX
9703f42c0c
backend_firefox: bump max supported schema version 2025-06-24 08:47:51 +08:00
CismonX
b9f1f74c69
doc: improve docs
- Remove some useless babbles.
- Better pseudo-code examples for ioctls.
- Wording and structuring improvements.
- ...
2025-06-23 21:19:47 +08:00
CismonX
52c48c92b8
backend_firefox: use wal mode in backend_mkfs()
This also allows us to reinstate the sandboxed db_check(), which
was removed in commits 35d4a93a41
and 0fd2cbbc9d.
2025-06-22 12:44:42 +08:00
CismonX
ffa28a4091
test: fetch prng seed from environment variable
Tests that require PRNG are by default seeded from `/dev/urandom`.
However, the user should be able to override with a given seed
to reproduce a failing test.

Instead of messing with command-line arguments, a cleaner approach
is to fetch the seed from an environment variable.
2025-06-19 16:51:35 +08:00
CismonX
6c38779e78
prng: debug-print seed if fetched from getrandom() 2025-06-19 16:49:41 +08:00
CismonX
72d55ed68f
test: do not skip test if a backend is disabled
For `ATX_CHECK_FS()` and `ATX_CHECK_FS_NEW()`, do not skip the test
if a backend (or its "-write" feature) is disabled, since
`exit 77` skips the entire test group, not current `AT_CHECK()`.

Instead, silently pass the current test.
2025-06-19 08:02:03 +08:00
CismonX
dad1984d7f
build: move bookmarkfs_util.pc.in to ./src
Whatever pkg-config is interested in all lies within ./src.
2025-06-18 16:13:07 +08:00
CismonX
9de097d9ee
build: refactor configuration scripts
- Move common `AC_DEFINE()` usage to `EX_FEAT()`.
- Remove unused argument `action-if-disabled` for `EX_FEAT()`.
- Conditionally enable features by default.
2025-06-17 13:41:17 +08:00
CismonX
fb0d39dfc3
defs: remove HAVE_PIPE2 macro
This feature check is only used in `xpipe2()`.
2025-06-17 12:24:08 +08:00
CismonX
99ac2b79f8
test: misc fix and refactor 2025-06-16 18:32:36 +08:00
CismonX
ba9a23c295
backend_firefox: refactor db query utils
- `DO_QUERY()` macro:
  * Pass stmt idx as argument instead of pointer.
  * Remove the `BEFORE_QUERY` argument.
- Shorter names for stmt idx.
2025-06-16 11:22:53 +08:00
CismonX
31f4311419
backend_firefox: misc refactor
- `mozorigin_get()`, `mozplace_addref()`: pass struct as argument.
- `parse_mozurl_host()`: rename to `parse_mozurl`; add comments;
  store parse results to argument of type `struct mozorigin`.
- Rename objects of type `struct mozXXX` from `XXX_cols` to `mX`.
- ...
2025-06-15 17:45:52 +08:00
CismonX
6fc165ff65
all: properly handle time_t on 32-bit platforms
Do not force 64-bit `time_t` on 32-bit platforms, since libfuse
does not do so.  Linking shared objects with incompatible types
breaks ABI, resulting in undefined behavior.

Instead, add run-time checks to make sure that timestamps do not
overflow.  If they do, set to `INT32_MAX`.

Also tidy up build scripts, tests, and the installation guide.
2025-06-14 11:43:03 +08:00
CismonX
f1451d206e
chore: bump version to 0.1.2 2025-06-12 11:31:39 +08:00
CismonX
303c934894
test: misc refactor
- If the utility library is not built, link to the existing library
  when building helper programs for filesystem tests.
- Refactor PRNG seeding in tests.
2025-06-11 20:45:16 +08:00
CismonX
14b5a79147
build: fix builds with existing utility library
Regression in commit 81db7a786c.
2025-06-11 17:11:19 +08:00
CismonX
1e5149cdc0
doc: misc corrections and improvements 2025-06-10 21:17:37 +08:00
CismonX
6f2a00dd6f
fs_ops: refactor setattr
- Only perform one operation per call.
- Always fail with EPERM if given unsupported flags.
2025-06-09 19:51:25 +08:00
CismonX
c23623b5a3
test: add tests for file atime/mtime 2025-06-08 12:34:10 +08:00
CismonX
fdfd6fe069
backend_firefox: fix directory attr update 2025-06-08 07:24:39 +08:00
CismonX
5082e7c67c
fs_ops: fix mtime update of regular files
Explicit mtime update (e.g., via futimens(2)) should override
file handle mtime.
2025-06-07 19:55:27 +08:00
CismonX
72b9e200d9
fs_ops: misc refactor
- Use nodeid for fh_map keys as well as bm_*() and intfs_*() args.
- bm_readdir(), fs_op_rename(): check subsys type more gracefully.
- ...
2025-06-07 08:57:48 +08:00
CismonX
29338ca02b
backend_firefox: purge dangling refs on delete
When a bookmark is deleted, if there are no other bookmarks
referencing the corresponding `moz_place` entry, tag and keyword
references to that entry are considered "dangling" references,
and shall be automatically removed.

Also reverts commit b5fa6960ef,
since the NULL title check is no longer necessary.
2025-06-06 22:46:44 +08:00
CismonX
eb426f9fc4
bookmarkctl: xattr-get: rename -m option to -a 2025-06-06 19:41:09 +08:00
CismonX
b5fa6960ef
backend_firefox: hide dangling keywords
When a bookmark associated with a keyword is deleted,
there may still be dangling references (e.g., tags) to the
corresponding `moz_places` entry.

By filtering out NULL titles, `bookmark_lookup()` and
`bookmark_list()` now only give keywords associated with
valid bookmarks.
2025-06-05 08:27:59 +08:00
CismonX
6fb9438d3c
fs_ops: fix opendir for keyword directory 2025-06-05 07:39:53 +08:00
CismonX
c1cf9db2a1
test: add tests for tags and keywords 2025-06-02 09:20:19 +08:00
CismonX
495f8592e6
fs_ops: fix deletion of tag directories 2025-06-02 08:41:33 +08:00
CismonX
d5eae85774
test: refactor filesystem tests 2025-06-01 21:35:07 +08:00
CismonX
ab88e0e839
test: add tests for directory entries 2025-05-28 18:51:02 +08:00
CismonX
e6809b7e84
xstd: add xgetdents() 2025-05-28 18:27:55 +08:00
CismonX
5a28b069c4
fs_ops: fix file size opened with O_CREAT|O_TRUNC
Also make sure that new regular files always have a size of zero.
2025-04-29 14:48:39 +08:00
CismonX
83f201435f
backend_firefox: add keyword xattr
This allows users to quickly discover which keyword is
associated with a given bookmark.

Updating keywords via xattr is not implemented,
since it can be done trivially using existing API.
2025-04-29 12:46:29 +08:00
CismonX
0e20604c73
backend_firefox: fix stmt id for keyword delete 2025-04-28 20:04:40 +08:00
CismonX
9c0d5fb337
test: improve filesystem tests
- Add a final check to see whether the fs daemon is still there.
- Other misc updates.
2025-04-07 19:32:44 +08:00
CismonX
565063ee9b
chore: bump version to 0.1.1 2025-04-07 12:23:50 +08:00
36 changed files with 1559 additions and 1001 deletions

View file

@ -166,19 +166,6 @@ Notes
In this case, the utility library will not be built from source,
and other components will link to the specified library instead.
### Targeting 32-bit Platforms
BookmarkFS requires 64-bit `off_t` and `time_t`, which may not be supported
on 32-bit platforms (e.g., i386 FreeBSD does not support 64-bit `time_t`).
If using Autoconf 2.72 or later, the configuration script automatically
performs checks and defines necessary macros, and fails if unsupported.
With legacy Autoconf, only `off_t` is checked.
To manually configure 64-bit `time_t`, add preprocessor flags
`-D_TIME_BITS=64` (or something equivalent, depending on the toolchain).
If unsupported, `make` will fail.
### FreeBSD and GNU libiconv
NOTE: You may skip this section if _not_ building the Chromium backend.

View file

@ -8,5 +8,3 @@
ACLOCAL_AMFLAGS = -I m4
SUBDIRS = doc src tests
pkgconfig_DATA = bookmarkfs_util.pc

View file

@ -8,8 +8,8 @@ dnl This file is offered as-is, without any warranty.
dnl
AC_PREREQ([2.70])
AC_INIT([bookmarkfs], [0.1.0], [bug-bookmarkfs@nongnu.org])
AC_CONFIG_SRCDIR([bookmarkfs_util.pc.in])
AC_INIT([bookmarkfs], [0.1.2], [bug-bookmarkfs@nongnu.org])
AC_CONFIG_SRCDIR([src/bookmarkfs_util.pc.in])
AC_CONFIG_HEADERS([config.h])
AC_CONFIG_MACRO_DIR([m4])
AC_CONFIG_TESTDIR([tests])
@ -40,81 +40,50 @@ AC_PROG_MAKE_SET
# -- Checks for features --
EX_FEAT([bookmarkfs-util], [no], [the BookmarkFS utility library])
EX_FEAT([bookmarkfs-util], [no], [the BookmarkFS utility library], , [1])
EX_FEAT([bookmarkctl], [no], [the bookmarkctl program])
EX_FEAT([bookmarkctl], [no], [the bookmarkctl program], , [1])
EX_FEAT([bookmarkfs-fsck], [no], [the fsck.bookmarkfs program], [
AS_VAR_SET([enable_bookmarkfs_util], [yes])
])
AS_VAR_SET([enable_bookmarkfs_util], [yes])
AS_VAR_SET([enable_interactive_fsck], [yes])
], [1])
EX_FEAT([bookmarkfs-mkfs], [no], [the mkfs.bookmarkfs program])
EX_FEAT([bookmarkfs-mkfs], [no], [the mkfs.bookmarkfs program], , [1])
EX_FEAT([bookmarkfs-mount], [no], [the mount.bookmarkfs program], [
AS_VAR_SET([enable_bookmarkfs_util], [yes])
])
], [1])
EX_FEAT([sandbox], [yes], [sandboxing], [
AC_DEFINE([BOOKMARKFS_SANDBOX], [1],
[Define to 1 if sandboxing is enabled.])
])
EX_FEAT([sandbox], [yes], [sandboxing])
AS_VAR_IF([host_os_is_linux], [yes], [
EX_FEAT([sandbox-landlock], [yes], [Landlock features for sandboxing], [
AC_DEFINE([BOOKMARKFS_SANDBOX_LANDLOCK], [1],
[Define to 1 if Landlock is enabled for sandboxing.])
])
EX_FEAT([sandbox-landlock], [yes], [Landlock features for sandboxing])
])
EX_FEAT([xxhash-inline], [no], [using xxhash as a header-only library], [
AC_DEFINE([BOOKMARKFS_XXHASH_INLINE], [1],
[Define to 1 if using xxhash as a header-only library.])
])
EX_FEAT([xxhash-inline], [no], [using xxhash as a header-only library])
EX_FEAT([bookmarkfs-debug], [no], [debugging features for BookmarkFS], [
AC_DEFINE([BOOKMARKFS_DEBUG], [1],
[Define to 1 if BookmarkFS debugging is enabled.])
])
EX_FEAT([bookmarkfs-debug], [no], [debugging features for BookmarkFS])
EX_FEAT([backend-firefox], [no], [Firefox backend], [
AS_VAR_SET([enable_bookmarkfs_util], [yes])
])
AS_VAR_SET([enable_bookmarkfs_util], [yes])
AS_VAR_SET([enable_backend_firefox_write], [yes])
], [1])
EX_FEAT([backend-firefox-write], [yes],
[write support for the Firefox backend],
[
AC_DEFINE([BOOKMARKFS_BACKEND_FIREFOX_WRITE], [1],
[Define to 1 if the Firefox backend supports writing.])
])
EX_FEAT([backend-firefox-write], , [write support for the Firefox backend])
EX_FEAT([backend-chromium], [no], [Chromium backend], [
AS_VAR_SET([enable_bookmarkfs_util], [yes])
])
AS_VAR_SET([enable_bookmarkfs_util], [yes])
AS_VAR_SET([enable_backend_chromium_write], [yes])
], [1])
EX_FEAT([backend-chromium-write], [yes],
[write support for the Chromium backend],
[
AC_DEFINE([BOOKMARKFS_BACKEND_CHROMIUM_WRITE], [1],
[Define to 1 if the Chromium backend supports writing.])
])
EX_FEAT([backend-chromium-write], , [write support for the Chromium backend])
EX_FEAT([native-watcher], [yes], [platform-specific file watcher], [
AC_DEFINE([BOOKMARKFS_NATIVE_WATCHER], [1],
[Define to 1 if platform-specific file watcher is enabled.])
])
EX_FEAT([native-watcher], [yes], [platform-specific file watcher])
EX_FEAT([interactive-fsck], [yes],
[interactive features for fsck.bookmarkfs],
[
AS_VAR_IF([enable_bookmarkfs_fsck], [no], [
AS_VAR_SET([enable_interactive_fsck], [no])
], [
AC_DEFINE([BOOKMARKFS_INTERACTIVE_FSCK], [1],
[Define to 1 if interactive fsck features are enabled.])
])
])
EX_FEAT([interactive-fsck], , [interactive features for fsck.bookmarkfs])
EX_FEAT([fsck-handler-tcl], [no], [Tcl-based fsck handler])
EX_FEAT([fsck-handler-tcl], [no], [Tcl-based fsck handler], , [1])
# -- Checks for libraries --
@ -196,18 +165,7 @@ AS_VAR_IF([ac_have_largefile], [no], [
]))
])
m4_version_prereq([2.72], [
AC_SYS_YEAR2038
AS_VAR_IF([ac_have_year2038], [no], [
AC_MSG_ERROR(m4_normalize([
64-bit time_t is unsupported on this platform.
]))
])
], [
dnl fallback to compile-time check...
AX_COMPILE_CHECK_SIZEOF([time_t])
])
AX_COMPILE_CHECK_SIZEOF([time_t])
AX_COMPILE_CHECK_SIZEOF([uintptr_t], [#include <stdint.h>])
# -- Output --
@ -217,8 +175,7 @@ AC_DEFINE([BUILDING_BOOKMARKFS], [1], [Define to 1 if building BookmarkFS.])
AC_CONFIG_FILES([
Makefile
doc/Makefile
src/Makefile
src/Makefile src/bookmarkfs_util.pc
tests/Makefile tests/atlocal
bookmarkfs_util.pc
])
AC_OUTPUT

View file

@ -32,7 +32,7 @@ bookmarkctl - manage a mounted BookmarkFS filesystem
.B bookmarkctl
.B xattr\-get
.RI [ options ]
.B \-m
.B \-a
.IR attrname "... " pathname
.PP
.B bookmarkctl
@ -145,7 +145,7 @@ Treat the value as binary, and print it verbatim.
.IP
If this option is not provided, non-printable characters are replaced with '?'.
.TP
.B \-m
.B \-a
Switch to multi-attrname mode, where multiple extended attribute names
can be specified instead of multiple files.
.IP

View file

@ -38,7 +38,7 @@
@end macro
@macro tcldoc {name, path}
@uref{https://www.tcl.tk/man/tcl8.6.13/\path\, \name\}
@uref{https://www.tcl.tk/man/tcl8.6.16/\path\, \name\}
@end macro
@copying
@ -106,22 +106,19 @@ for CI build logs.
Currently, BookmarkFS only runs on GNU/Linux and FreeBSD.
Although BookmarkFS sticks hard to POSIX and avoids using platform-specific
Although BookmarkFS sticks hard to POSIX and avoids platform-specific
features, porting it to other operating systems is not trivial.
The major pitfall is the @linuxdoc{FUSE, filesystems/fuse.html} dependency.
Generally speaking, FUSE is Linux-only.
FreeBSD partially implements the FUSE protocol in its kernel,
to the extent that BookmarkFS is mostly usable.
However, that's not the case for other operating systems.
FUSE was originally Linux-only.
Recent versions of the FreeBSD kernel partially implements the FUSE protocol,
however, that's not the case for other operating systems.
For example, OpenBSD implements its own FUSE protocol,
which is incompatible with the Linux one.
OpenBSD does provide a libfuse-compatible library, however,
it only covers the high-level API, while BookmarkFS uses the
@uref{https://libfuse.github.io/doxygen/fuse__lowlevel_8h.html, low-level API}.
For a similar reason, @uref{https://github.com/winfsp/winfsp, WinFsp}
won't work if you're trying to port BookmarkFS to Microsoft Windows.
Other notable portability issues:
@ -129,13 +126,10 @@ Other notable portability issues:
@item Sandboxing
Not all operating system kernels provide sandboxing mechanisms similar to
Linux and FreeBSD.
@xref{Sandboxing}.
If not supported, operations that require sandboxing should fail.
Users should not be provided with a false sense of security.
If they wish, they could pass a @option{-o no_sandbox} option to
explicitly disable sandboxing.
Also @pxref{Sandboxing}.
If unsupported, operations that require sandboxing should fail,
unless sandboxing is explicitly disabled by the caller.
@end table
@ -209,11 +203,8 @@ the effective user ID and group ID of the calling process.
On FreeBSD, you may wish to set the @samp{vfs.usermount}
@freebsdmanpage{sysctl, 8} to @t{1} for unprivileged mounts.
To unmount a BookmarkFS filesystem, run @linuxmanpage{fusermount3, 1} or
@linuxmanpage{umount, 8} on @var{target}.
The daemon process will automatically dismount the filesystem upon
@code{SIGINT} or @code{SIGTERM} receipt, however, it only works in
non-sandbox mode.
To unmount a BookmarkFS filesystem, run @linuxmanpage{fusermount3, 1}
(with the @option{-u} option) or @linuxmanpage{umount, 8} on @var{target}.
Options:
@ -255,32 +246,32 @@ access the files.
@item -o ctime
Maintain last status change time instead of last modification time.
Usually, a bookmark's ``modification time'' attribute behaves differently
from both mtime and ctime.
From a browser's perspective, the ``modification time'' attribute
of a bookmark behaves differently from both mtime and ctime.
In Chromium, for instance, when a bookmark is renamed, neither itself
nor the parent directory changes timestamp accordingly.
BookmarkFS do not follow browser behavior here, and instead try to stay
compatible with POSIX.
In BookmarkFS, mtime behavior mostly stays compatible with POSIX,
with a few caveats.
Since a bookmark has only one ``modification time'' attribute instead of two,
the user has to choose which one to maintain:
@table @asis
@item Last modification time
ctime only updates when mtime does.
@item Last modification time (default)
mtime updates normally; ctime value always follow mtime.
The kernel may automatically update and cache ctime,
making it appear more ``correct'' than expected.
However, this behavior should not be relied upon.
@item Last status change time
ctime updates normally; mtime always updates when ctime does,
ctime updates normally; mtime value always follow ctime,
even if the file content is not modified.
This behavior may be inefficient, but makes applications that depend on ctime
less fragile.
@end table
The kernel may automatically update and cache ctime,
making it more ``correct'' than what we expect.
However, this behavior should not be relied upon.
@item -o eol
Add a newline (ASCII LF character) to the end of each file.
@ -302,8 +293,8 @@ Do not use Landlock for sandboxing.
This option is ignored on non-Linux platforms.
Without Landlock, sandboxing offers less security.
However, Landlock is a rather new feature (requires kernel version 5.13
or later), thus we provide an option to disable it separately.
Nonetheless, we provide an option to disable it separately,
since Landlock is a rather new feature (requires kernel version 5.13 or later).
@item -F
Stay in the foreground, do not daemonize.
@ -364,7 +355,7 @@ so that the user don't have to manually dismount the inactive filesystem.
See @linuxmanpage{mount.fuse3, 8} for details.
This option is helpful when sandboxing is enabled, especially when
the @option{-F} option is given, since a sandboxed process itself can neither
given the @option{-F} option, since a sandboxed process itself can neither
@linuxmanpage{umount, 2} nor fork-exec.
Currently, this option is not available on FreeBSD.
@ -630,7 +621,7 @@ Displays extended attribute values.
@example
bookmarkctl xattr-get [@var{options}] @var{attrname} @var{pathname}...
bookmarkctl xattr-get [@var{options}] -m @var{attrname}... @var{pathname}
bookmarkctl xattr-get [@var{options}] -a @var{attrname}... @var{pathname}
@end example
@table @var
@ -650,7 +641,7 @@ Treat the value as binary, and print it verbatim.
If this option is not provided, non-printable characters are replaced with
@samp{?}.
@item -m
@item -a
Switch to multi-attrname mode, where multiple extended attribute names
can be specified instead of multiple files.
@ -822,8 +813,8 @@ However, consecutive lookups and @code{readdir()}s should produce consistent
results for that file, provided that it is not renamed or deleted.
Tag files behave differently from traditional hard links.
If the original bookmark file is renamed or deleted,
it may also change accordingly.
If the associated bookmark file of a tag is renamed or deleted,
it may change accordingly.
It may even link to another file that was previously shadowed.
Applications should tread lightly if they wish to cache tag directory entries.
@ -887,14 +878,13 @@ for web browser bookmarks.
An unexpected internal error occurred, likely due to a bug in BookmarkFS,
or a corruption in the bookmark storage.
When this error occurs, a log message describing the situation may be printed
to the standard error of the filesystem daemon.
Sometimes this error comes from the kernel-side FUSE implementation,
and there's no error message.
Once this error occurs, behavior of any further operations on the filesystem
is undefined.
A log message describing the situation may be printed to the standard error
of the filesystem daemon.
Sometimes the error comes from the kernel, and there's no error message.
@item ESTALE
The file associated with the file descriptor no longer exists.
@ -994,15 +984,17 @@ To change the order of directory entries, @pxref{Permute Directory Entries}.
BookmarkFS provides an I/O control for rearranging directory entries:
@example c
#include <bookmarkfs/ioctl.h>
struct bookmarkfs_permd_data permd_data = @{ /* ... */ @};
int ioctl (int dirfd, BOOKMARKFS_IOC_PERMD,
struct bookmarkfs_permd_data const *argp);
status = ioctl(dirfd, BOOKMARKFS_IOC_PERMD, &permd_data);
// ...
@end example
The @code{bookmarkfs_permd_data} structure is defined as:
@example c
#include <bookmarkfs/ioctl.h>
struct bookmarkfs_permd_data @{
enum bookmarkfs_permd_op op;
@ -1086,25 +1078,25 @@ from another bookmark management software.
@node Online Filesystem Check
@subsection Online Filesystem Check
To perform filesystem check on a mounted BookmarkFS filesystem,
use the following I/O controls:
BookmarkFS provides I/O controls to perform online filesystem checks:
@example c
#include <bookmarkfs/ioctl.h>
struct bookmarkfs_fsck_data fsck_data = @{ /* ... */ @};
int ioctl (int dirfd, BOOKMARKFS_IOC_FSCK_NEXT,
struct bookmarkfs_fsck_data *argp);
int ioctl (int dirfd, BOOKMARKFS_IOC_FSCK_APPLY,
struct bookmarkfs_fsck_data *argp);
int ioctl (int dirfd, BOOKMARKFS_IOC_FSCK_REWIND);
result = ioctl(dirfd, BOOKMARKFS_IOC_FSCK_NEXT, &fsck_data);
// ...
result = ioctl(dirfd, BOOKMARKFS_IOC_FSCK_APPLY, &fsck_data);
// ...
result = ioctl(dirfd, BOOKMARKFS_IOC_FSCK_REWIND);
// ...
@end example
@anchor{Filesystem-Check Data}
The @code{bookmarkfs_fsck_data} structure is defined as:
@example c
#include <bookmarkfs/ioctl.h>
struct bookmarkfs_fsck_data @{
uint64_t id;
uint64_t extra;
@ -1120,14 +1112,14 @@ Find the next bookmark with invalid name under the directory.
@anchor{Filesystem-Check Result Code}
@cindex Filesystem-Check Result Code
On success, @code{ioctl()} updates @var{argp}, and returns
On success, @code{ioctl()} updates @var{fsck_data}, and returns
one of the values defined in @code{enum bookmarkfs_fsck_result}:
@table @code
@item BOOKMARKFS_FSCK_RESULT_END
There are no more bookmarks with invalid name under the directory.
Fields in @var{argp} have unspecified values.
Fields in @var{fsck_data} have unspecified values.
@item BOOKMARKFS_FSCK_RESULT_NAME_DUPLICATE
The bookmark name duplicates with another bookmark which appears earlier
@ -1166,7 +1158,7 @@ otherwise the behavior is undefined.
The @code{name} field should be set to the new name for the bookmark.
The @code{extra} field is unused.
On success, @code{ioctl()} updates @var{argp}, and returns
On success, @code{ioctl()} updates @var{fsck_data}, and returns
one of the values defined in @code{enum bookmarkfs_fsck_result},
like with @code{BOOKMARKFS_IOC_FSCK_NEXT}.
Additionally, it may also return:
@ -1252,7 +1244,7 @@ The Firefox backend provides access to the bookmark data of the web browser
and its derivatives, notably @uref{https://www.torproject.org/, Tor Browser}
and @uref{https://librewolf.net/, Librewolf}.
Backend name (for the @option{-o backend} option): @samp{firefox}.
Backend name (for the @option{-o backend=@var{name}} option): @samp{firefox}.
Firefox bookmarks are stored in a SQLite database under the profile directory.
When mounting the filesystem, this pathname shall be passed as the @var{src}
@ -1358,6 +1350,12 @@ The bookmark creation time.
Value is a decimal integer representing number of microseconds since
the Unix epoch.
@item keyword
The keyword associated with the bookmark.
@xref{Keywords}.
This attribute is read-only.
@end table
Notable limitations:
@ -1367,6 +1365,9 @@ Notable limitations:
of a bookmark may affect that of other bookmarks,
due to the way bookmarks are associated with the URLs.
@item For a similar reason, directory atime is not supported, since the atime
attribute only exists on URLs, and directories are not associated with URLs.
@item The bookmark storage (SQLite database file) should be writable even if
the BookmarkFS filesystem is mounted read-only, due to the limitations
of WAL-mode.
@ -1382,7 +1383,7 @@ The Chromium backend provides access to the bookmark data of the web browser
notably @uref{https://github.com/ungoogled-software/ungoogled-chromium,
ungoogled-chromium}.
Backend name (for the @option{-o backend} option): @samp{chromium}.
Backend name (for the @option{-o backend=@var{name}} option): @samp{chromium}.
Chromium bookmarks are stored in a text file (in JSON format)
under the profile directory.
@ -1492,7 +1493,7 @@ Notable limitations:
@itemize @bullet{}
@item Does not scale well with large bookmark storage,
since everything is kept in memory, and the entire JSON file has to be parsed
whenever a reload is required.
(with a parser known not to be fast) whenever a reload is required.
@item No support for tags (@pxref{Tags}) and keywords (@pxref{Keywords}),
since Chromium does not have such concepts.
@ -1723,7 +1724,7 @@ as well as a list of backend-specific options.
Indicates that the function should print a version message.
Mutually exclusive with @code{BOOKMARKFS_BACKEND_INFO_HELP}.
In addition to the version number, the version message may also contain
In addition to the version number, the version message may contain
dependency versions, compile-time options, and other information
that can be used to determine a specific version of the backend.
@ -1964,7 +1965,7 @@ The pointer referring to the backend context.
@subsection Enter Sandbox
The @code{backend_sandbox} function is called to instruct the backend to
enter a sandboxed state where it has limited access to system resources,
enter a sandboxed state, where it has limited access to system resources,
thereby improving security.
@xref{Sandboxing}.
@ -1973,10 +1974,10 @@ is @emph{not} set for this context.
Considering how sandboxing is usually implemented,
it may be complicated or even impractical for multiple backend contexts
to enter sandbox separately without affecting other ones.
to enter sandbox separately without affecting one another.
Thus it is guaranteed that, when launched from a frontend program like
@command{mount.bookmarkfs}, only one backend context will be created
throughout the lifetime of the process if sandboxing is ever needed.
@command{mount.bookmarkfs} which requires sandboxing, only one
backend context will be created throughout the lifetime of the process.
Type of the @code{backend_sandbox} function is defined as:
@ -4301,27 +4302,7 @@ Refer to the source code in @file{src/sandbox.c} for implementation details.
@node File Watcher
@section File Watcher
File watchers detect filesystem changes using platform-specific features:
@table @asis
@item Linux
Implemented with @linuxmanpage{fanotify, 7}.
Requires kernel version 5.13 or later for unprivileged users.
@linuxmanpage{inotify, 7} does not have this limitation, however,
it is incompatible with our sandboxing design.
@item FreeBSD
Implemented with the @code{EVFILT_VNODE} filter of @freebsdmanpage{kevent, 2}.
@item Fallback
When no platform-specific filesystem watching mechanism is available,
periodically checks the @code{st_ino} and @code{st_mtim} attributes
of the watched file with @posixfuncmanpage{fstatat}.
This approach is less efficient than ``native'' implementations,
but should work on any POSIX-compatible system.
@end table
Each file watcher watches for changes of a single file on the filesystem.
Functions:
@ -4354,7 +4335,26 @@ A bit array of the following flags:
@table @asis
@item @code{WATCHER_FALLBACK}
Do not use platform-specific features.
By default, the file watcher uses platform-specific API to detect
filesystem changes:
@table @asis
@item Linux
Implemented with @linuxmanpage{fanotify, 7}.
Requires kernel version 5.13 or later for unprivileged users.
@linuxmanpage{inotify, 7} does not have this limitation, however,
it is incompatible with our sandboxing design.
@item FreeBSD
Implemented with the @code{EVFILT_VNODE} filter of @freebsdmanpage{kevent, 2}.
@end table
With this flag, the watcher instead periodically checks the @code{st_ino} and
@code{st_mtim} attributes of the watched file with @posixfuncmanpage{fstatat}.
The fallback implementation is less efficient than ``native'' ones,
but should work on any POSIX-compatible system.
@item @code{WATCHER_NOOP}
The watcher does nothing, and @code{watcher_poll()} always return

View file

@ -8,8 +8,8 @@ dnl This file is offered as-is, without any warranty.
dnl
dnl
dnl EX_FEAT(feature, default-value, description, [action-if-enabled],
dnl [action-if-disabled])
dnl EX_FEAT(feature, [default-value], description, [action-if-enabled],
dnl [no-ac-define])
dnl
dnl Provide an option to enable or disable a feature.
dnl
@ -19,15 +19,19 @@ AC_DEFUN([EX_FEAT], [
AC_MSG_CHECKING(m4_normalize([if $3 is enabled]))
AC_ARG_ENABLE([$1], m4_normalize([
AS_HELP_STRING([--]arg_action_[-$1], arg_action_ [$3])
]), , [
]), , m4_ifnblank([$2], [
AS_VAR_SET([enable_]feat_name_, [$2])
])
AS_VAR_IF([enable_]feat_name_, [no], [
AC_MSG_RESULT([no])
$5
], [
]))
AS_VAR_IF([enable_]feat_name_, [yes], [
AC_MSG_RESULT([yes])
$4
m4_ifblank([$5], [
AC_DEFINE(m4_if(m4_substr(feat_name_, 0, 10), [bookmarkfs], ,
[BOOKMARKFS_])[]m4_toupper(feat_name_),
[1], [Define to 1 if $3 is enabled.])
])
], [
AC_MSG_RESULT([no])
])
AS_VAR_SET([desc_]feat_name_, ["$3"])
m4_popdef([arg_action_])

View file

@ -13,6 +13,7 @@ noinst_HEADERS = backend_util.h db.h defs.h frontend_util.h fs_ops.h \
xattr.h xstd.h
lib_LTLIBRARIES =
pkglib_LTLIBRARIES =
pkgconfig_DATA = bookmarkfs_util.pc
BASE_CPPFLAGS_ =
if NO_FILE_NAME
@ -28,7 +29,7 @@ UTIL_LIBS_ =
if BOOKMARKFS_UTIL
UTIL_LIBS_ += libbookmarkfs_util.la
else
UTIL_LIBS_ += $(BOOKMARKFS_LIBS)
UTIL_LIBS_ += $(BOOKMARKFS_UTIL_LIBS)
endif # BOOKMARKFS_UTIL
BACKEND_SOURCES_ = backend_util.c lib.c xstd.c

View file

@ -58,17 +58,14 @@
#include "watcher.h"
#include "xstd.h"
#if defined(SIZEOF_TIME_T) && (SIZEOF_TIME_T != 8)
# error "64-bit time_t is required"
#endif
#define BACKEND_FILENAME_GUID ( 1u << 16 )
// Chromium uses Windows FILETIME epoch instead of Unix epoch.
// Chromium uses Windows FILETIME epoch, which is
// `((1970 - 1601) * 365 + 89) * 24 * 3600` seconds before the Unix epoch.
//
// See Chromium source code: /base/time/time.h
// (`base::Time::kTimeTToMicrosecondsOffset`)
#define EPOCH_DIFF ( (time_t)((1970 - 1601) * 365 + 89) * 24 * 3600 )
#define EPOCH_DIFF INT64_C(11644473600)
#define BOOKMARKS_ROOT_ID 0
@ -387,7 +384,7 @@ build_tsnode (
ts = &now;
}
time_t secs = ts->tv_sec + EPOCH_DIFF;
int64_t secs = ts->tv_sec + EPOCH_DIFF;
int64_t microsecs = secs * 1000000 + ts->tv_nsec / 1000;
char buf[32];
@ -1461,12 +1458,17 @@ parse_ts (
}
if (buf != NULL) {
time_t secs = microsecs / 1000000;
if (unlikely(secs < EPOCH_DIFF)) {
int64_t secs = microsecs / 1000000 - EPOCH_DIFF;
if (unlikely(secs < 0)) {
// Stay away from negative tv_sec
secs = EPOCH_DIFF;
secs = 0;
}
buf->tv_sec = secs - EPOCH_DIFF;
#if defined(SIZEOF_TIME_T) && (SIZEOF_TIME_T < 8)
else if (secs > INT32_MAX) {
secs = INT32_MAX;
}
#endif
buf->tv_sec = secs;
buf->tv_nsec = (microsecs % 1000000) * 1000;
}
return 0;

File diff suppressed because it is too large Load diff

View file

@ -48,7 +48,7 @@ struct xattr_get_ctx {
char eol;
unsigned binary : 1;
unsigned multi_name : 1;
unsigned multi_attr : 1;
};
// Forward declaration start
@ -249,8 +249,8 @@ subcmd_xattr_get (
ctx.binary = 1;
break;
}
OPT_OPT('m') {
ctx.multi_name = 1;
OPT_OPT('a') {
ctx.multi_attr = 1;
break;
}
OPT_OPT('q') {
@ -269,7 +269,7 @@ subcmd_xattr_get (
return -1;
}
if (ctx.multi_name) {
if (ctx.multi_attr) {
return xattr_get_one(argv[argc - 1], argv, argc - 1, &ctx);
}
for (int i = 1; i < argc; ++i) {
@ -416,7 +416,7 @@ xattr_get_one (
for (int i = 0; i < names_cnt; ++i) {
char const *name = names[i];
ctx->prefix = ctx->multi_name ? name : path;
ctx->prefix = ctx->multi_attr ? name : path;
status = bookmarkfs_xattr_get(fd, name, xattr_get_cb, ctx);
if (status < 0) {
goto end;

View file

@ -92,10 +92,6 @@
# define UNUSED_VAR(name) name##_unused_ VARATTR_UNUSED_
#endif /* defined(HAVE_STDC_23) */
#if defined(__FreeBSD__) || (defined(__linux__) && defined(_GNU_SOURCE))
# define HAVE_PIPE2 1
#endif
#ifndef __FreeBSD__
# define O_RESOLVE_BENEATH 0
# define PROT_MAX(prot) 0

File diff suppressed because it is too large Load diff

View file

@ -28,10 +28,8 @@
#include <stdio.h>
#include <stdlib.h>
#include <dirent.h>
#include <fcntl.h>
#include <sys/stat.h>
#include <sys/syscall.h>
#include <unistd.h>
#include "backend.h"
@ -67,10 +65,6 @@ struct fsck_dir {
#define FSCK_DIR_DONE ( 1u << 0 )
// Forward declaration start
#ifdef __linux__
static ssize_t getdents_ (int, void *, size_t);
#endif
static int next_subdir (struct fsck_ctx *, struct fsck_dir *,
struct dirent const **);
static int open_subdir (int, char const *, uint64_t *);
@ -79,22 +73,6 @@ static void print_help (void);
static void print_version (void);
// Forward declaration end
#ifdef __linux__
// Some libc (e.g., musl) may declare a getdents() function
// in dirent.h with conflicting types.
#define getdents getdents_
static ssize_t
getdents_ (
int dirfd,
void *buf,
size_t bufsize
) {
return syscall(SYS_getdents64, dirfd, buf, bufsize);
}
#endif /* defined(__linux__) */
static int
next_subdir (
struct fsck_ctx *ctx,
@ -112,7 +90,7 @@ next_subdir (
ctx->dent_buf = xrealloc(ctx->dent_buf, ctx->dent_buf_size);
}
ssize_t nbytes
= getdents(dir->fd, ctx->dent_buf + start, DIRENT_BUFSIZE);
= xgetdents(dir->fd, ctx->dent_buf + start, DIRENT_BUFSIZE);
if (nbytes < 0) {
log_printf("getdents(): %s", xstrerror(errno));
return -1;

View file

@ -29,6 +29,7 @@
#include "prng.h"
#include <errno.h>
#include <inttypes.h>
#include <string.h>
#include <sys/random.h>
@ -89,5 +90,8 @@ prng_seed (
// as many bytes as requested.
// This is guaranteed on both Linux and FreeBSD.
debug_assert(nbytes == sizeof(state));
debug_printf("prng seed: "
"%016" PRIx64 "%016" PRIx64 "%016" PRIx64 "%016" PRIx64,
state[0], state[1], state[2], state[3]);
return 0;
}

View file

@ -27,7 +27,7 @@
#define BOOKMARKFS_VER_MAJOR 0
#define BOOKMARKFS_VER_MINOR 1
#define BOOKMARKFS_VER_PATCH 0
#define BOOKMARKFS_VER_PATCH 2
#define bookmarkfs_make_vernum(major, minor, patch) \
( ((major) << 16) | ((minor) << 8) | ((patch) << 0) )

View file

@ -111,14 +111,14 @@ xpipe2 (
int pipefd[2],
int flags
) {
#ifdef HAVE_PIPE2
#if defined(__FreeBSD__) || (defined(__linux__) && defined(_GNU_SOURCE))
if (0 != pipe2(pipefd, flags)) {
log_printf("pipe2(): %s", xstrerror(errno));
return -1;
}
return 0;
#else /* !defined(HAVE_PIPE2) */
#else
if (0 != pipe(pipefd)) {
log_printf("pipe(): %s", xstrerror(errno));
return -1;
@ -147,7 +147,7 @@ xpipe2 (
close(pipefd[1]);
return -1;
#endif /* defined(HAVE_PIPE2) */
#endif
}
void *

View file

@ -26,6 +26,9 @@
#include <stdio.h>
#include <time.h>
#include <dirent.h>
#include <sys/syscall.h>
#include "defs.h"
#ifdef HAVE___BUILTIN_EXPECT
@ -84,6 +87,12 @@
} while (0)
#endif
#if defined(__linux__)
# define xgetdents(fd, buf, bufsz) syscall(SYS_getdents64, fd, buf, bufsz)
#elif defined(__FreeBSD__)
# define xgetdents(fd, buf, bufsz) getdents(fd, buf, bufsz)
#endif
/**
* Prints a message to standard error, and then aborts.
*/

View file

@ -9,7 +9,8 @@
EXTRA_DIST = package.m4 testsuite.at $(TESTSUITE) $(TESTS_)
TESTS_ = lib_hash.at lib_prng.at lib_watcher.at lib_sandbox.at \
lib_hashmap.at fs_basic.at fs_regrw.at
lib_hashmap.at fs_basic.at fs_regrw.at fs_dents.at fs_assoc.at \
fs_times.at
# Helper programs for testing
@ -30,12 +31,13 @@ if BOOKMARKFS_MOUNT
check_fs_CPPFLAGS = -I$(top_srcdir)/src
check_fs_LDADD =
check_fs_SOURCES = check_fs.c
check_fs_SOURCES = check_fs.c check_fs_dents.c check_fs_regrw.c \
check_fs_times.c check_util.c
if BOOKMARKFS_UTIL
check_fs_CPPFLAGS += -DHAVE_BOOKMARKFS_UTIL
check_fs_LDADD += $(top_builddir)/src/libbookmarkfs_util.la
check_fs_SOURCES += check_fs_regrw.c check_util.c
check_fs_LDADD += $(top_builddir)/src/libbookmarkfs_util.la
else
check_fs_LDADD += $(BOOKMARKFS_UTIL_LIBS)
endif # BOOKMARKFS_UTIL
endif # BOOKMARKFS_MOUNT

View file

@ -54,10 +54,12 @@ dispatch_subcmds (
status = subcmd_ismount(argc, argv);
} else if (0 == strcmp("sleep", cmd)) {
status = subcmd_sleep(argc, argv);
#ifdef HAVE_BOOKMARKFS_UTIL
} else if (0 == strcmp("regrw", cmd)) {
status = check_fs_regrw(argc, argv);
#endif
} else if (0 == strcmp("dents", cmd)) {
status = check_fs_dents(argc, argv);
} else if (0 == strcmp("times", cmd)) {
status = check_fs_times(argc, argv);
} else {
log_printf("bad subcmd '%s'", cmd);
}

272
tests/check_fs_dents.c Normal file
View file

@ -0,0 +1,272 @@
/**
* bookmarkfs/tests/check_fs_dents.c
* ----
*
* Copyright (C) 2025 CismonX <admin@cismon.net>
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
#ifdef HAVE_CONFIG_H
# include "config.h"
#endif
#include <errno.h>
#include <stdlib.h>
#include <string.h>
#include <fcntl.h>
#include <sys/stat.h>
#include <unistd.h>
#include "check_util.h"
#include "frontend_util.h"
#include "ioctl.h"
#include "prng.h"
#define ITEM_DELETED ( 1u << 0 )
#define ITEM_DIRTY ( 1u << 1 )
#define ITEM_MARKED ( 1u << 2 )
struct check_item {
int id;
unsigned flags;
int ref;
};
// Forward declaration start
static int dent_check (int, struct check_item *, int, int);
static int dent_delete (int, struct check_item *);
static int dent_new (int, struct check_item *);
static int dent_permute (int, struct check_item *, struct check_item *);
static int do_check_fs_dents (int, int);
// Forward declaration end
static int
dent_check (
int dirfd,
struct check_item *items,
int n,
int ignore_dirty
) {
char buf[4096];
struct check_item const *last_found = NULL;
for (ssize_t off = 0, len = 0; ; ) {
if (off == len) {
len = xgetdents(dirfd, buf, sizeof(buf));
if (len < 0) {
log_printf("getdents(): %s", strerror(errno));
return -1;
}
if (len == 0) {
break;
}
off = 0;
}
struct dirent *dent = (struct dirent *)(buf + off);
off += dent->d_reclen;
int id = atoi(dent->d_name);
if (id < 0 || id >= n) {
return -1;
}
struct check_item *found = items + items[id].ref;
if (found->flags & (ITEM_DELETED | ITEM_MARKED)) {
return -1;
}
if (ignore_dirty && found->flags & ITEM_DIRTY) {
continue;
}
if (last_found != NULL && found <= last_found) {
return -1;
}
last_found = found;
found->flags |= ITEM_MARKED;
}
for (last_found = items + n; items < last_found; ++items) {
if (ignore_dirty && items->flags & ITEM_DIRTY) {
continue;
}
if (!(items->flags & (ITEM_DELETED | ITEM_MARKED))) {
return -1;
}
items->flags &= ~ITEM_MARKED;
}
return 0;
}
static int
dent_delete (
int dirfd,
struct check_item *item
) {
if (item->flags & ITEM_DELETED) {
return 0;
}
char name[16];
sprintf(name, "%d", item->id);
if (0 != unlinkat(dirfd, name, 0)) {
log_printf("unlinkat(): %s", strerror(errno));
return -1;
}
item->flags |= ITEM_DELETED;
return 0;
}
static int
dent_new (
int dirfd,
struct check_item *item
) {
char name[16];
sprintf(name, "%d", item->id);
int fd = openat(dirfd, name, O_CREAT | O_EXCL, 0600);
if (fd < 0) {
log_printf("openat(): %s", strerror(errno));
return -1;
}
close(fd);
item->flags &= ~ITEM_DELETED;
return 0;
}
static int
dent_permute (
int dirfd,
struct check_item *item1,
struct check_item *item2
) {
int op = BOOKMARKFS_PERMD_OP_SWAP;
struct check_item *itemx = item2;
if ((item1->flags & ITEM_DELETED) || (item2->flags & ITEM_DELETED)) {
if (item1->flags & ITEM_DELETED) {
struct check_item *item_tmp = item1;
item1 = item2;
item2 = item_tmp;
}
if (item1 > item2) {
op = BOOKMARKFS_PERMD_OP_MOVE_BEFORE;
for (itemx = item2 + 1; itemx->flags & ITEM_DELETED; ++itemx);
} else {
op = BOOKMARKFS_PERMD_OP_MOVE_AFTER;
for (itemx = item2 - 1; itemx->flags & ITEM_DELETED; --itemx);
}
}
struct bookmarkfs_permd_data permd_data;
permd_data.op = op;
sprintf(permd_data.name1, "%d", item1->id);
sprintf(permd_data.name2, "%d", itemx->id);
if (0 != ioctl(dirfd, BOOKMARKFS_IOC_PERMD, &permd_data)) {
log_printf("ioctl(): %s", strerror(errno));
return -1;
}
struct check_item item_tmp = *item1;
item1->id = item2->id;
item1->flags = item2->flags | ITEM_DIRTY;
item2->id = item_tmp.id;
item2->flags = item_tmp.flags | ITEM_DIRTY;
return 0;
}
static int
do_check_fs_dents (
int dirfd,
int n
) {
#define ASSERT_EQ(val, expr) ASSERT_EXPR_INT(expr, r_, (val) == r_, goto end;)
#define ASSERT_NE(val, expr) ASSERT_EXPR_INT(expr, r_, (val) != r_, goto end;)
struct check_item *items = calloc(n, sizeof(struct check_item));
if (items == NULL) {
return -1;
}
int status = -1;
for (int i = 0; i < n; ++i) {
struct check_item *item = items + i;
item->ref = item->id = i;
ASSERT_EQ(0, dent_new(dirfd, item));
}
ASSERT_EQ(0, lseek(dirfd, 0, SEEK_SET));
for (int i = 0; i < n / 4; ++i) {
uint64_t bits = prng_rand();
struct check_item *i1 = &items[bits % n];
struct check_item *i2 = &items[(bits >> 32) % n];
if (i1 == i2 || bits >> 63) {
ASSERT_EQ(0, dent_delete(dirfd, i1));
ASSERT_EQ(0, dent_delete(dirfd, i2));
#if !defined(__FreeBSD__)
} else {
if (i1->flags & ITEM_DELETED && i2->flags & ITEM_DELETED) {
continue;
}
ASSERT_EQ(0, dent_permute(dirfd, i1, i2));
items[i1->id].ref = i1 - items;
items[i2->id].ref = i2 - items;
#endif
}
}
ASSERT_EQ(0, dent_check(dirfd, items, n, 1));
ASSERT_EQ(0, lseek(dirfd, 0, SEEK_SET));
ASSERT_EQ(0, dent_check(dirfd, items, n, 0));
status = 0;
end:
free(items);
return status;
}
int
check_fs_dents (
int argc,
char *argv[]
) {
int n = -1;
OPT_START(argc, argv, "n:")
OPT_OPT('n') {
n = atoi(optarg);
break;
}
OPT_END
if (n <= 0) {
log_printf("bad size %d", n);
return -1;
}
if (argc < 1) {
log_puts("path not given");
return -1;
}
char const *path = argv[0];
if (0 != prng_seed_from_env()) {
return -1;
}
int dirfd = open(path, O_RDONLY | O_DIRECTORY);
if (dirfd < 0) {
log_printf("open: %s: %s", path, strerror(errno));
return -1;
}
int status = do_check_fs_dents(dirfd, n);
close(dirfd);
return status;
}

View file

@ -122,7 +122,7 @@ do_check_fs_regrw (
#ifndef O_DIRECT
# define O_DIRECT 0
#endif
int fd = open(path, O_RDWR | O_TRUNC | O_DIRECT);
int fd = open(path, O_RDWR | O_CREAT | O_TRUNC | O_DIRECT, 0600);
ASSERT_NE(-1, fd);
struct stat stat_buf;
@ -167,21 +167,13 @@ check_fs_regrw (
int argc,
char *argv[]
) {
uint64_t seed_buf[4], *seed = NULL;
int file_max = -1;
OPT_START(argc, argv, "n:s:")
OPT_START(argc, argv, "n:")
OPT_OPT('n') {
file_max = atoi(optarg);
break;
}
OPT_OPT('s') {
if (0 != prng_seed_from_hex(seed_buf, optarg)) {
return -1;
}
seed = seed_buf;
break;
}
OPT_END
if (file_max <= 0 || file_max % sizeof(uint64_t) != 0) {
@ -194,7 +186,7 @@ check_fs_regrw (
}
char const *path = argv[0];
if (0 != prng_seed(seed)) {
if (0 != prng_seed_from_env()) {
return -1;
}
return do_check_fs_regrw(path, file_max);

193
tests/check_fs_times.c Normal file
View file

@ -0,0 +1,193 @@
/**
* bookmarkfs/tests/check_fs_times.c
* ----
*
* Copyright (C) 2025 CismonX <admin@cismon.net>
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
#ifdef HAVE_CONFIG_H
# include "config.h"
#endif
#include <errno.h>
#include <string.h>
#include <time.h>
#include <fcntl.h>
#include <sys/stat.h>
#include <unistd.h>
#include "check_util.h"
#include "backend_util.h"
#include "frontend_util.h"
#include "prng.h"
// Forward declaration start
static int compare_timespec (struct timespec, struct timespec);
static int do_check_fs_times (int, int);
static void usecs_to_timespec (struct timespec *, uint64_t);
// Forward declaration end
static int
compare_timespec (
struct timespec ts1,
struct timespec ts2
) {
#define timespec_to_usecs(ts) ( (ts).tv_sec * 1000000 + (ts).tv_nsec / 1000 )
int64_t usecs1 = timespec_to_usecs(ts1);
int64_t usecs2 = timespec_to_usecs(ts2);
if (usecs1 == usecs2) {
return 0;
}
return usecs2 > usecs1 ? 1 : -1;
}
static int
do_check_fs_times (
int dirfd,
int rounds
) {
#define FILE1_NAME "foo.tmp"
#define FILE2_NAME "bar.tmp"
#define ASSERT_EQ(val, expr) ASSERT_EXPR_INT(expr, r_, (val) == r_, goto end;)
#define ASSERT_NE(val, expr) ASSERT_EXPR_INT(expr, r_, (val) != r_, goto end;)
int status = -1;
int fd = -1;
struct timespec now;
ASSERT_EQ(0, clock_gettime(CLOCK_REALTIME, &now));
fd = openat(dirfd, FILE1_NAME, O_WRONLY | O_CREAT | O_EXCL, 0600);
ASSERT_NE(-1, fd);
struct stat stat_buf;
ASSERT_EQ(0, fstat(fd, &stat_buf));
ASSERT_NE(-1, compare_timespec(now, stat_buf.st_mtim));
ASSERT_EQ(0, fstat(dirfd, &stat_buf));
ASSERT_NE(-1, compare_timespec(now, stat_buf.st_mtim));
now = stat_buf.st_mtim;
ASSERT_EQ(11, write(fd, "foo:bar/baz", 11));
ASSERT_EQ(0, fstat(fd, &stat_buf));
ASSERT_NE(-1, compare_timespec(now, stat_buf.st_mtim));
ASSERT_EQ(0, renameat(dirfd, FILE1_NAME, dirfd, FILE2_NAME));
ASSERT_EQ(0, fstat(dirfd, &stat_buf));
ASSERT_NE(-1, compare_timespec(now, stat_buf.st_mtim));
now = stat_buf.st_mtim;
ASSERT_EQ(0, ftruncate(fd, 10));
ASSERT_EQ(0, fstat(fd, &stat_buf));
ASSERT_NE(-1, compare_timespec(now, stat_buf.st_mtim));
struct timespec times[2];
times[0].tv_nsec = UTIME_OMIT;
times[1].tv_nsec = UTIME_NOW;
ASSERT_EQ(0, futimens(dirfd, times));
ASSERT_EQ(0, fstat(dirfd, &stat_buf));
ASSERT_NE(-1, compare_timespec(now, stat_buf.st_mtim));
now = stat_buf.st_mtim;
times[0].tv_nsec = UTIME_NOW;
times[1].tv_nsec = UTIME_OMIT;
ASSERT_EQ(0, futimens(fd, times));
ASSERT_EQ(0, fstat(fd, &stat_buf));
ASSERT_NE(-1, compare_timespec(now, stat_buf.st_atim));
ASSERT_EQ(0, clock_gettime(CLOCK_REALTIME, &now));
ASSERT_NE(-1, compare_timespec(stat_buf.st_atim, now));
for (int i = 0; i < rounds; ++i) {
usecs_to_timespec(&times[0], prng_rand());
usecs_to_timespec(&times[1], prng_rand());
ASSERT_EQ(0, futimens(fd, times));
ASSERT_EQ(0, fstat(fd, &stat_buf));
ASSERT_EQ(0, compare_timespec(times[0], stat_buf.st_atim));
ASSERT_EQ(0, compare_timespec(times[1], stat_buf.st_mtim));
}
ASSERT_EQ(0, unlinkat(dirfd, FILE2_NAME, 0));
ASSERT_EQ(0, fstat(dirfd, &stat_buf));
ASSERT_NE(-1, compare_timespec(now, stat_buf.st_mtim));
ASSERT_EQ(0, clock_gettime(CLOCK_REALTIME, &now));
ASSERT_NE(-1, compare_timespec(stat_buf.st_mtim, now));
status = 0;
end:
if (fd >= 0) {
close(fd);
}
return status;
}
static void
usecs_to_timespec (
struct timespec *ts_buf,
uint64_t usecs
) {
int64_t secs = usecs / 1000000;
#if defined(SIZEOF_TIME_T) && (SIZEOF_TIME_T < 8)
secs %= (int64_t)INT32_MAX + 1;
#else
secs %= TIMESPEC_SEC_MAX + 1;
#endif
ts_buf->tv_sec = secs;
ts_buf->tv_nsec = (usecs % 1000000) * 1000;
}
int
check_fs_times (
int argc,
char *argv[]
) {
int rounds = -1;
OPT_START(argc, argv, "r:")
OPT_OPT('r') {
rounds = atoi(optarg);
break;
}
OPT_END
if (rounds < 0) {
log_printf("bad rounds cnt %d", rounds);
return -1;
}
if (argc < 1) {
log_puts("path not given");
return -1;
}
char const *path = argv[0];
if (0 != prng_seed_from_env()) {
return -1;
}
int dirfd = open(path, O_RDONLY | O_DIRECTORY);
if (dirfd < 0) {
log_printf("open: %s: %s", path, strerror(errno));
return -1;
}
int status = do_check_fs_times(dirfd, rounds);
close(dirfd);
return status;
}

View file

@ -168,18 +168,10 @@ check_hashmap (
int argc,
char *argv[]
) {
uint64_t seed_buf[4], *seed = NULL;
int size_exp = -1;
int rounds = -1;
OPT_START(argc, argv, "s:n:r:")
OPT_OPT('s') {
if (0 != prng_seed_from_hex(seed_buf, optarg)) {
return -1;
}
seed = seed_buf;
break;
}
OPT_START(argc, argv, "n:r:")
OPT_OPT('n') {
size_exp = atoi(optarg);
break;
@ -199,7 +191,7 @@ check_hashmap (
return -1;
}
if (0 != prng_seed(seed)) {
if (0 != prng_seed_from_env()) {
return -1;
}
return do_check_hashmap(1u << size_exp, rounds);

View file

@ -104,24 +104,16 @@ subcmd_prng (
int argc,
char *argv[]
) {
uint64_t seed_buf[4], *seed = NULL;
int n = 0;
OPT_START(argc, argv, "s:n:")
OPT_OPT('s') {
if (0 != prng_seed_from_hex(seed_buf, optarg)) {
return -1;
}
seed = seed_buf;
break;
}
OPT_START(argc, argv, "n:")
OPT_OPT('n') {
n = atoi(optarg);
break;
}
OPT_END
if (0 != prng_seed(seed)) {
if (0 != prng_seed_from_env()) {
return -1;
}
for (; n > 0; --n) {

View file

@ -58,18 +58,17 @@ do_check_sandbox (
# error "not implemented"
#endif
#define ASSERT_BAD_SYS(expr, cleanup_action) \
ASSERT_EXPR_INT(expr, r_, (err_ = errno, r_ < 0), { \
cleanup_action \
goto end; \
}); \
ASSERT_EXPR_INT(err_, r_, r_ == ERR1 || r_ == ERR2, goto end;)
#define ASSERT_BAD_SYS(expr, cleanup_action) \
ASSERT_EXPR_INT(expr, r_, r_ < 0, { \
cleanup_action \
goto end; \
}); \
ASSERT_EXPR_INT(errno, r_, r_ == ERR1 || r_ == ERR2, goto end;)
#define ASSERT_BAD_FD(expr) ASSERT_BAD_SYS(expr, close(r_);)
#define ASSERT_EQ(val, expr) ASSERT_EXPR_INT(expr, r_, (val) == r_, goto end;)
#define ASSERT_NE(val, expr) ASSERT_EXPR_INT(expr, r_, (val) != r_, goto end;)
int err_;
int status = -1;
int fd = socket(AF_INET, SOCK_STREAM, 0);

View file

@ -26,18 +26,25 @@
#include <inttypes.h>
#include <stdio.h>
#include <stdlib.h>
#include "prng.h"
int
prng_seed_from_hex (
uint64_t *buf,
char const *str
) {
int cnt = sscanf(str,
prng_seed_from_env (void)
{
char const *seed_str = getenv("BOOKMARKFS_TEST_PRNG_SEED");
if (seed_str == NULL) {
return prng_seed(NULL);
}
uint64_t seed[4];
int cnt = sscanf(seed_str,
"%16" SCNx64 "%16" SCNx64 "%16" SCNx64 "%16" SCNx64,
&buf[0], &buf[1], &buf[2], &buf[3]);
&seed[0], &seed[1], &seed[2], &seed[3]);
if (cnt != 4) {
log_printf("bad seed '%s'", str);
log_puts("bad prng seed");
return -1;
}
return 0;
return prng_seed(seed);
}

View file

@ -35,6 +35,18 @@
action_if_false \
} while (0)
int
check_fs_dents (
int argc,
char *argv[]
);
int
check_fs_times (
int argc,
char *argv[]
);
int
check_fs_regrw (
int argc,
@ -60,9 +72,6 @@ check_watcher (
);
int
prng_seed_from_hex (
uint64_t *buf,
char const *str
);
prng_seed_from_env (void);
#endif /* !defined(BOOKMARKFS_CHECK_UTIL_H_) */

65
tests/fs_assoc.at Normal file
View file

@ -0,0 +1,65 @@
dnl
dnl Copyright (C) 2025 CismonX <admin@cismon.net>
dnl
dnl Copying and distribution of this file, with or without modification,
dnl are permitted in any medium without royalty,
dnl provided the copyright notice and this notice are preserved.
dnl This file is offered as-is, without any warranty.
dnl
AT_SETUP([fs: tags and keywords])
AT_KEYWORDS([fs assoc tag keyword])
ATX_CHECK_FS_NEW_ASSOC([eol], , [
ATX_RUN_REPEAT([8], [
name=$(ath_fn_rand_u64_hex)
tag=$(ath_fn_rand_u64_hex)
keyword=$(ath_fn_rand_u64_hex)
content=foo:$(ath_fn_rand_u64_hex)
ATX_RUN([
echo "$content/1" > $name-1
echo "$content/2" > $name-2
echo "$content/3" > $name-3
mkdir "$atx_tags/$tag-1" "$atx_tags/$tag-2" "$atx_tags/$tag-3"
ln $name-1 $name-2 "$atx_tags/$tag-1"
ln $name-2 $name-3 "$atx_tags/$tag-2"
ln $name-3 $name-1 "$atx_tags/$tag-3"
test $name-1 -ef "$atx_tags/$tag-1/$name-1"
test $name-1 -ef "$atx_tags/$tag-3/$name-1"
test $name-2 -ef "$atx_tags/$tag-1/$name-2"
test $name-2 -ef "$atx_tags/$tag-2/$name-2"
test $name-3 -ef "$atx_tags/$tag-3/$name-3"
test $name-3 -ef "$atx_tags/$tag-2/$name-3"
ln $name-1 "$atx_keywords/$keyword-2"
ln $name-2 "$atx_keywords/$keyword-3"
ln $name-3 "$atx_keywords/$keyword-1"
test $name-1 -ef "$atx_keywords/$keyword-2"
test $name-2 -ef "$atx_keywords/$keyword-3"
test $name-3 -ef "$atx_keywords/$keyword-1"
rm "$atx_tags/$tag-1/$name-1"
rm "$atx_tags/$tag-2/$name-2"
rm "$atx_tags/$tag-3/$name-3"
test ! -e "$atx_tags/$tag-1/$name-1"
test ! -e "$atx_tags/$tag-2/$name-2"
test ! -e "$atx_tags/$tag-3/$name-3"
rm "$atx_keywords/$keyword-1"
rm "$atx_keywords/$keyword-2"
test ! -e "$atx_keywords/$keyword-1"
test ! -e "$atx_keywords/$keyword-2"
rm $name-1 $name-2 $name-3
test ! -e "$atx_tags/$tag-1/$name-2"
test ! -e "$atx_tags/$tag-2/$name-3"
test ! -e "$atx_tags/$tag-3/$name-1"
test ! -e "$atx_keywords/$keyword-3"
rmdir "$atx_tags/$tag-1" "$atx_tags/$tag-2" "$atx_tags/$tag-3"
])
])
])
AT_CLEANUP

View file

@ -15,53 +15,48 @@ AT_KEYWORDS([fs basic])
ATX_CHECK_FS_NEW_ANY([eol], , [
ATX_RUN_REPEAT([8], [
name=$(ath_fn_rand_u64_hex)
name_1=${name}_1
name_2=${name}_2
content=foo:$(ath_fn_rand_u64_hex)
content_1=${content}/1
content_2=${content}/2
ATX_RUN([
echo "$content_1" > $name_1
test "$(cat $name_1)" = "$content_1"
echo "$content_2" > $name_2
test "$(cat $name_2)" = "$content_2"
echo "$content/1" > $name-1
test "$(cat $name-1)" = "$content/1"
echo "$content/2" > $name-2
test "$(cat $name-2)" = "$content/2"
mv $name_1 $name_2
test ! -e $name_1
test "$(cat $name_2)" = "$content_1"
mv $name-1 $name-2
test ! -e $name-1
test "$(cat $name-2)" = "$content/1"
mv $name_2 $name_1
test ! -e $name_2
test "$(cat $name_1)" = "$content_1"
mv $name-2 $name-1
test ! -e $name-2
test "$(cat $name-1)" = "$content/1"
mkdir $name_2
mv $name_1 $name_2/$name_2
test ! -e $name_1
test "$(cat $name_2/$name_2)" = "$content_1"
mkdir $name-2
mv $name-1 $name-2/$name-2
test ! -e $name-1
test "$(cat $name-2/$name-2)" = "$content/1"
! mkdir $name_2/$name_2
mkdir $name_2/$name_1
mv $name_2/$name_2 $name_2/$name_1/$name_1
test "$(cat $name_2/$name_1/$name_1)" = "$content_1"
! mkdir $name-2/$name-2
mkdir $name-2/$name-1
mv $name-2/$name-2 $name-2/$name-1/$name-1
test "$(cat $name-2/$name-1/$name-1)" = "$content/1"
mkdir $name_1
! mv $name_1 $name_2/$name_1/$name_1
! mv $name_1 $name_2
mkdir $name-1
! mv $name-1 $name-2/$name-1/$name-1
! mv $name-1 $name-2
! mv $name_2/$name_1/$name_1 $name_2
rm $name_2/$name_1/$name_1
test ! -e $name_2/$name_1/$name_1
! mv $name-2/$name-1/$name-1 $name-2
rm $name-2/$name-1/$name-1
test ! -e $name-2/$name-1/$name-1
mv $name_1 $name_2
test ! -e $name_1
test -d $name_2/$name_1
mv $name-1 $name-2
test ! -e $name-1
test -d $name-2/$name-1
! rmdir $name_2
rmdir $name_2/$name_1
rmdir $name_2
test ! -e $name_2
! rmdir $name-2
rmdir $name-2/$name-1
rmdir $name-2
test ! -e $name-2
])
])
])

24
tests/fs_dents.at Normal file
View file

@ -0,0 +1,24 @@
dnl
dnl Copyright (C) 2025 CismonX <admin@cismon.net>
dnl
dnl Copying and distribution of this file, with or without modification,
dnl are permitted in any medium without royalty,
dnl provided the copyright notice and this notice are preserved.
dnl This file is offered as-is, without any warranty.
dnl
AT_SETUP([fs: directory entries])
AT_KEYWORDS([fs dents])
ATX_CHECK_FS_NEW_ANY(, , [
ATX_RUN_REPEAT([8], [
name=$(ath_fn_rand_u64_hex)
mkdir $name
ATX_RUN([
check-fs dents -n 1024 $name
])
])
])
AT_CLEANUP

View file

@ -10,17 +10,11 @@ dnl
AT_SETUP([fs: regular file read/write])
AT_KEYWORDS([fs regrw])
ATX_CHECK_FS_NEW_ANY([file_max=524288], [
# requires PRNG
ATX_FEAT_PREREQ([bookmarkfs-util])
], [
ATX_CHECK_FS_NEW_ANY([file_max=524288], , [
name=$(ath_fn_rand_u64_hex)
seed=$(ath_fn_prng_seed)
echo "prng seed: $seed"
ATX_RUN([
touch $name
check-fs regrw -n 524288 -s "$seed" $name
check-fs regrw -n 524288 $name
])
])

22
tests/fs_times.at Normal file
View file

@ -0,0 +1,22 @@
dnl
dnl Copyright (C) 2025 CismonX <admin@cismon.net>
dnl
dnl Copying and distribution of this file, with or without modification,
dnl are permitted in any medium without royalty,
dnl provided the copyright notice and this notice are preserved.
dnl This file is offered as-is, without any warranty.
dnl
AT_SETUP([fs: last access/modification times])
AT_KEYWORDS([fs times])
ATX_CHECK_FS_NEW_ANY(, , [
name=$(ath_fn_rand_u64_hex)
mkdir $name
ATX_RUN([
check-fs times -r 512 $name
])
])
AT_CLEANUP

View file

@ -11,12 +11,6 @@ AT_SETUP([util lib: hashmap])
AT_KEYWORDS([lib hashmap])
ATX_CHECK_LIB([
seed="${CHECK_HASHMAP_PRNG_SEED}"
if test -z "$seed"; then
seed=$(ath_fn_prng_seed)
fi
echo "prng seed: $seed"
size="${CHECK_HASHMAP_DATA_SIZE}"
if test -z "$size"; then
# Requires at least 128 MiB of memory on a 64-bit platform.
@ -29,7 +23,7 @@ ATX_CHECK_LIB([
fi
ATX_RUN([
check-util-lib hashmap -s "$seed" -n "$size" -r "$rounds"
check-util-lib hashmap -n "$size" -r "$rounds"
])
])

View file

@ -15,13 +15,16 @@ AT_KEYWORDS([lib prng])
# For reliable testing of PRNGs, see <https://prng.di.unimi.it/#quality>.
ATX_CHECK_LIB([
gen_num() {
check-util-lib prng -s$1 -n$2
env BOOKMARKFS_TEST_PRNG_SEED=$1 check-util-lib prng -n$2
}
ATX_RUN_REPEAT([16], [
seed=$(ath_fn_prng_seed)
repeat=1
if test -z "$BOOKMARKFS_TEST_PRNG_SEED"; then
repeat=16
fi
ATX_RUN_REPEAT([$repeat], [
seed=$(check-util-lib prng -n4 | tr -d '\n')
count=32
echo "prng seed: $seed"
num_1=$(gen_num $seed $count)
num_2=$(gen_num $seed $count)

View file

@ -125,8 +125,9 @@ dnl Check for a BookmarkFS filesystem.
dnl
m4_define([ATX_CHECK_FS], [
ATX_CHECK_SIMPLE([
ATX_FEAT_PREREQ([bookmarkfs-mount], [backend-$1])
ATX_FEAT_PREREQ([bookmarkfs-mount])
$5
ATX_FEAT_IF([backend-$1], , [exit])
"$buildsrcdir/mount.bookmarkfs" -F \
-o "backend=ATX_OPT_MODULE([backend_$1])" \
-o "fsname=check-$1,no_sandbox,$2" \
@ -139,6 +140,7 @@ m4_define([ATX_CHECK_FS], [
fi
done
$6
ATX_RUN_ONE([check-fs ismount "$4"])
], [
umount "$4"
])
@ -152,8 +154,9 @@ dnl created with mkfs.bookmarkfs.
dnl
m4_define([ATX_CHECK_FS_NEW], [
ATX_CHECK_FS([$1], [rw,$2], [bookmarks-$1], [$3], [
ATX_FEAT_PREREQ([bookmarkfs-mkfs], [backend-$1-write])
ATX_FEAT_PREREQ([bookmarkfs-mkfs])
$4
ATX_FEAT_IF([backend-$1-write], , [exit])
"$buildsrcdir/mkfs.bookmarkfs" \
-o "backend=ATX_OPT_MODULE([backend_$1]),force" \
"bookmarks-$1" || exit 1
@ -173,6 +176,17 @@ m4_define([ATX_CHECK_FS_NEW_ANY], [
])
])
dnl
dnl ATX_CHECK_FS_NEW_ASSOC([options], [prepare], [check])
dnl
m4_define([ATX_CHECK_FS_NEW_ASSOC], [
ATX_CHECK_FS_NEW([firefox], [$1], [mnt.tmp], [$2], [
atx_tags=../../tags
atx_keywords=../../keywords
ATX_RUN_IN_DIR([mnt.tmp/bookmarks/unfiled], [$3])
])
])
dnl -- Helper functions --
AT_TEST_HELPER_FN([rand_u64_hex], , , [
@ -185,15 +199,6 @@ AT_TEST_HELPER_FN([rand_base64], , , [
echo $(head -c$1 /dev/urandom | base64 -w0)
])
AT_TEST_HELPER_FN([prng_seed], , , [
seed_1=$(ath_fn_rand_u64_hex)
seed_2=$(ath_fn_rand_u64_hex)
seed_3=$(ath_fn_rand_u64_hex)
seed_4=$(ath_fn_rand_u64_hex)
echo $seed_1$seed_2$seed_3$seed_4
])
dnl -- Test groups --
AT_BANNER([The Utility Library])
@ -206,3 +211,6 @@ m4_include([lib_hashmap.at])
AT_BANNER([The Filesystem])
m4_include([fs_basic.at])
m4_include([fs_regrw.at])
m4_include([fs_dents.at])
m4_include([fs_assoc.at])
m4_include([fs_times.at])