# Changelog
All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## 0.12.0 (2024-02-28)
### Chore
- Use the cargo workspace package table
This allows for inheritance of common fields from the workspace root.
The following fields have been made common:
- authors
- license
- repository
- homepage
- edition
- Appease clippy unused imports
- tracefs review fixes
### Chore
- Don't use path deps in workspace
This moves the path dependencies back into the per-crate Cargo.toml.
It is required such that the release tooling can correctly calculate
which version constraints require changing when we perform a release.
### Documentation
- Document breaking changes
This provides a `BREAKING-CHANGES.md` that we can populate per-crate.
Doing so will allow us to pull this content into our changelog and
websites to make things easier for users.
- Add labels for optional features
Following the lead of crates like tokio and nix, we now annotate APIs
that require optional features. This helps in cases where a user wants
to have an `AsyncPerfEventArray` which is documented on crates.io, but
it's not obvious that you have to enable the `async` feature.
- Add crabby logo
- Document more breaking changes
- Add CHANGELOG
### New Features
- get_tracefs function
### Bug Fixes
- invalid transmute when calling fd
Corrent an invalid transmutation for sock_map.
fd is already a ref of MapFd, so transmuting &fd to &SockMapFd is
equivalent to transmuting &&SockMapFd into &SockMapFd which is buggy.
- Relax unnecessarily strict atomic ordering on probe event_alias
- remove useless `any` `all` in cfg.
### Other
- reformat to please rustfmt
- export some missing modules
Previously we were only re-exporting the program types from these, so
links and other pub types were not exported.
- perf_event: add inherit argument to attach()
- add StackTraceMap::remove()
- appease new nightly clippy lints
```
error: this call to `as_ref.map(...)` does nothing
--> aya/src/bpf.rs:536:30
|
536 | let btf_fd = btf_fd.as_ref().map(Arc::clone);
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: try: `btf_fd.clone()`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#useless_asref
note: the lint level is defined here
--> aya/src/lib.rs:41:5
|
41 | clippy::all,
| ^^^^^^^^^^^
= note: `#[deny(clippy::useless_asref)]` implied by `#[deny(clippy::all)]`
error: could not compile `aya` (lib) due to 1 previous error
warning: build failed, waiting for other jobs to finish...
error: initializer for `thread_local` value can be made `const`
--> aya/src/sys/fake.rs:14:61
|
14 | pub(crate) static TEST_MMAP_RET: RefCell<*mut c_void> = RefCell::new(ptr::null_mut());
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: replace with: `const { RefCell::new(ptr::null_mut()) }`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#thread_local_initializer_can_be_made_const
= note: `#[deny(clippy::thread_local_initializer_can_be_made_const)]` implied by `#[deny(clippy::all)]`
```
- appease nightly lint
```
error: lint `unused_tuple_struct_fields` has been renamed to `dead_code`
--> aya/src/lib.rs:74:5
|
74 | unused_tuple_struct_fields,
| ^^^^^^^^^^^^^^^^^^^^^^^^^^ help: use the new name: `dead_code`
|
= note: `-D renamed-and-removed-lints` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(renamed_and_removed_lints)]`
```
See https://github.com/rust-lang/rust/commit/9fcf9c141068984ffcbb4cb00c.
- add SchedClassifier::attach_to_link
Similar to Xdp::attach_to_link, can be used to replace/upgrade the
program attached to a link.
- add SchedClassifierLink::attach_type() getter
The link already exposes priority() and handle(). Expose attach_type()
too.
- Fix ringbuf docs
doctests are not running in CI and therefore the didn't catch the
ringbuf docs failures. This commit fixes the issues in the examples.
- pin for (async)perf_event_array
Implement pinning for perf_event_array and async_perf_event_array.
Additionally make the core MapData.pin method operate on a reference
rather than a mutable reference.
- make RingBuf: Send + Sync
There was no reason for them not to be -- the APIs all require mutable
references and hold onto mutable references, so there cannot be internal
concurrency. The !Send + !Sync came from the MMap, but not for any good
reason.
- extracting program and map names with the same function
- add MapInfo struct following the same pattern as ProgramInfo
This makes the APIs for loading maps and programs more similar.
- support loading a map by fd
This adds support to loading maps by fd similarly to the way programs
can be loaded by fd.
- make KernelVersion::code public
- Add markdownlint
This adds a linter to catch common markdown formatting errors.
The linter used is markdownlint-cli2 which is available on all platforms
and has an associated Github Action to automate these checks in CI.
Configuration is checked in at .markdownlint-cli2.yaml.
You may run the check locally using `markdownlint-cli2`.
Or you may install the extension for VSCode:
DavidAnson.vscode-markdownlint
- update async-io requirement from 1.3 to 2.0
Updates the requirements on [async-io](https://github.com/smol-rs/async-io) to permit the latest version.
- [Release notes](https://github.com/smol-rs/async-io/releases)
- [Changelog](https://github.com/smol-rs/async-io/blob/master/CHANGELOG.md)
- [Commits](https://github.com/smol-rs/async-io/compare/v1.3.0...v1.13.0)
---
updated-dependencies:
- dependency-name: async-io
dependency-type: direct:production
...
- fix unused async-io dependency linter error
Not using the `dep:` syntax created a Cargo feature flag for async-io,
though this feature alone does nothing without the `async_std` or
`async_tokio` features.
- Implement RingBuf
This implements the userspace binding for RingBuf.
Instead of streaming the samples as heap buffers, the process_ring
function takes a callback to which we pass the event's byte region,
roughly following [libbpf]'s API design. This avoids a copy and allows
marking the consumer pointer in a timely manner.
- move mmap from perf_buffer.rs to sys/mod.rs
mmap() is needed for the ring buffer implementation, so move it to a common module
- impl From for MapTypeError
- sort variants
Missed in 5e637071c130fece2b26f6a7246bdef5f782fced due to merge skew
with 7b71c7e1cd8d6948764d02afb0279151c6eae437.
- import types from std::ffi rather than libc
- sort variants
- remove redundant keys
`default-features = false` is already in the root Cargo.toml.
- add pin() api
- Adds new `maps_mut()` API to the BpfManager to allow us to iterate though
and pin all of maps at the same time.
- Adds new pin(Path)/unpin(Path) api to Maps so they
can be generically pinned AFTER load.
- Adds macro for pinning explicit map types in aya.
Convert all explicit map types "inner" field to be
pub crate in order to facilitate this.
- fix libbpf_pin_by_name
Aligns with libbpf for the special LIBBPF_PIN_BY_NAME
map flag. Specifically if the flag is provided without a pin path
default to "/sys/fs/bpf".
- Deprecate `syscall_prefix`
Using the prefix only for the host architecture is often not enough,
kernels usually provide symbols for more architectures, which are
used by multilib applications. Handling them might or might not be
necessary depending on the use case. Due to that complexity, we
decided to let the callers to handle prefixes the way they prefer.
- group_imports = "StdExternalCrate"
High time we stop debating this; let the robots do the work.
- Fix program loading on kernels with a patch > 255
- fix load time and add test
Time since boot is defined as the UNIX_EPOCH plus the duration
since boot. which is realtime - boottime NOT boottime - realtime.
Add a integration test to ensure this doesn't happen again.
- make maps work on kernels not supporting ProgIds
On startup, the kernel is probed for support of chained program ids for
CpuMap, DevMap and DevMapHash, and will patch maps at load time to have
the proper size. Then, at runtime, the support is checked and will error
out if a program id is passed when the kernel does not support it.
- use ProgramFd instead of impl AsRawFd
Not having a generic here allows to pass `None` without specifying the
actual type you don't care about.
- add documentation for XDP maps
- fix docstring missing trailing period
- add support for chained xdp programs in {cpu,dev}map
set/insert functions can now take an optional bpf program fd to run once
the packet has been redirected from the main probe
- add support for map-bound XDP programs
Such programs are to be bound to cpumap or devmap instead of the usual
network interfaces.
- Update XDP maps implementations
Map impls changed since this was first written.
- Implement XDP Map Types
This commit adds implementations for:
- xskmap
- devmap
- devmap_hash
- cpumap
Which can all be used to redirect XDP packets to various different
locations
- Make MapData::pin pub
This is to solve a use-case where a user (in this case bpfd) may want
to:
- MapData::from_pin to open a pinned map from bpffs
- MapData::pin to pin that object into another bpffs
Both operations should be easily accomplished without needing to cast
a MapData into a concrete Map type - e.g aya_nightly::maps::HashMap.
- Remove MapData::pinned
BPF objects can be pinned multiple times, to multiple different places.
Tracking whether or not a map is pinned in a bool is therefore not sufficient.
We could track this in a HashSet, but there is really no reason
to track it at all.
- fix typos, avoid fallible conversions
- MapData::{obj, fd} are private
- `MapFd` and `SockMapFd` are owned
- add program_info() api to program
Add a new api to the outer level `Program` structure which
allows users to get the program's kernel info before casting
it to an explicit program variant.
- support TryFrom for LRU hash maps
The macro to implement TryFrom for MapData didn't have the ability to
specify that more than one variant of MapData can be valid for a single
map implementation. Support for new syntax was added to the macro so that
the implementation can succeed for both valid variants in the HashMap
and PerCpuHashMap impl.
- rework TryFrom macros
The old macros were repetitive and inflexible. This unifies the various
macros used to generate TryFrom implementations for map implementations
from the relevant map enum variants.
Cleanup in anticipation of fixing #636.
The API changes are just about renaming the return to Self and
Self::Error; they are not real changes.
- access inner through async
Avoid holding onto raw file descriptors.
Remove some implied bounds (BorrowMut implies Borrow).
- ProgAttachLink and LircLink hold owned FDs
- use OwnedFd
- Use AsFd when attaching fds to programs
This is a breaking change but adds another level of safety to ensure
the file descriptor we receive is valid. Additionally, this allows
aya to internally easily duplicate this file descriptor using std
library methods instead of manually calling `dup` which doesn't
duplicate with the CLOSE_ON_EXEC flag that is standard pratice to
avoid leaking the file descriptor when exec'ing.
- Use BorrowedFd when using the program fd in sys/bpf.rs
This commit reveals but does not address a file descriptor leak in
LircLink2::query. This function returns a list of `LircLink`s where
each of them have a program file descriptor that is not going to be
closed. This commit does not add this leak; it merely makes it louder
in the code.
- support non-UTF8 probing
- avoid path UTF-8 assumptions
- deny various allow-by-default lints
- fix docs build
Appease the new lint rustdoc::redundant_explicit_links that was added in
https://github.com/rust-lang/rust/pull/113167.
- BloomFilter::insert takes &mut self
This is consistent with all the other maps.
- MapData::fd is non-optional
The primary driver of change here is that `MapData::create` is now a
factory function that returns `Result` rather than mutating
`&mut self`. The remaining changes are consequences of that change, the
most notable of which is the removal of several errors which are no
longer possible.
- use RAII to close FDs
- `ProgramData::attach_prog_fd` is owned
This prevents a file descriptor leak when extensions are used.
This is an API breaking change.
- `ProgramFd` is owned
- add helper methods for ProgramInfo
- Add helper methods to get useful information from the ProgramInfo
object which is returned by the `loaded_programs()` API. Specifically
this code mirrors the `bpftool prog` command in terms of useful fields.
- Add a new API macro to each aya `Program` type to allow us to fetch
its accompanying `ProgramInfo` metadata after its been loaded.
- Add a new ProgramInfo constructor that builds a new instance using
a raw fd.
- Add a smoke test for the loaded_programs() API as well as
all the relevant methods on the ProgramInfo type.
- Plug attach_btf_obj_fd leak
- Don't store bpf_fd in MapData
This is only used in create and therefore can be passed
as a parameter.
- refactor btf_obj_get_info_by_fd to share code
- add map_ids to bpf_prog_get_info_by_fd
Allows the caller to pass a slice which the kernel will populate with
map ids used by the program.
- avoid vector allocation when parsing ksyms
- Use OwnedFd in FdLink.
- Remove name from ProgramSection
The name here is never used as we get the program name from the symbol
table instead.
- refactor target resolution
This attempts to do fewer lossy conversions and to avoid some
allocations.
- extract library path resolving
The function is extracted so that a test could be written. This test is
valid on linux-gnu targets, and it doesn't need any special privileges.
This is in anticipation of removing the code that uses this functionality
(seemingly incidentally) from integration tests.
- Set BPF_F_SLEEPABLE for sleepable programs
- preallocate the vector
This code badly needs tests :(
- plug file descriptor leak
This leaked a file descriptor if bpf_prog_get_info_by_fd failed.
- push error construction up
- make `loaded_programs` opaque
- extract common SyscallError
We currently have 4 copies of this.
- `sys_bpf` takes mut ref
Some syscalls mutate the argument, we can't be passing an immutable
reference here.
- avoid repeating BPF_BTF_LOAD dance
- Return `OwnedFd` for `perf_event_open`.
This fixes a file descriptor leak when creating a link of
BPF_PERF_EVENT attach type.
- better panic messages
Always include operands in failing assertions. Use assert_matches over
manual match + panic.
- Use Arc when loading BTF fd
This fixes an existing file descriptor leak when there is BTF data in
the loaded object.
To avoid lifetime issues while having minimal impact to UX the
`OwnedFd` returned from the BPF_BTF_LOAD syscall will be wrapped in an
`Arc` and shared accross the programs and maps of the loaded BPF
file.
- Make SysResult generic on Ok variant
- bpf_prog_get_fd_by_id returns OwnedFd
- Define dependencies on the workspace level
This way we will avoid version mismatches and make differences in
features across our crates clearer.
- Ignore embedded BTF error if not truely required
This allows fallback to BTF manual relocation when BTF loading fail when not truely required.
- compile C probes using build.rs
- Add libbpf as a submodule. This prevents having to plumb its location
around (which can't be passed to Cargo build scripts) and also
controls the version against which codegen has run.
- Move bpf written in C to the integration-test crate and define
constants for each probe.
- Remove magic; each C source file must be directly enumerated in the
build script and in lib.rs.
- don't allocate static strings
- Make Features part of the public API
This commit adds a new probe for bpf_attach_cookie, which would be used
to implement USDT probes. Since USDT probes aren't currently supported,
we this triggers a dead_code warning in clippy.
There are cases where exposing FEATURES - our lazy static - is actually
helpful to users of the library. For example, they may wish to choose to
load a different version of their bytecode based on current features.
Or, in the case of an orchestrator like bpfd, we might want to allow
users to describe which features their program needs and return nice
error message is one or more nodes in their cluster doesn't support the
necessary feature set.
To do this without breaking the API, we make all the internal members of
the `Features` and `BtfFeatures` structs private, and add accessors for
them. We then add a `features()` API to avoid leaking the
lazy_static.
- Remove iter_key from LPM Trie API
Based on the discussion in Discord we've decided to drop the
iter_key() API for LPM Trie. According to the kernel self-tests and
experimentation done in Aya, providing a key into bpf_map_get_next_id
will either:
- If key is an EXACT match, proceed iterating through all keys in the
trie from this point
- If key is NOT an EXACT match, proceed iterating through all keys in
the trie starting at the leftmost entry.
An API in Aya could be crafted that gets the LPM match + less specific
matches for a prefix using these semantics BUT it would only apply to
userspace. Therefore we've opted out of fixing this.
- replace os::unix::prelude with os::fd
- allow global value to be optional
This allow to not error out when a global symbol is missing from the object.
- add syscall_prefix and syscall_fnname_add_prefix
These two functions are needed because kernel symbols representing
syscalls have architecture-specific prefixes.
These are the equivalent of bcc's get_syscall_fnname and
get_syscall_prefix.
- Fix uprobe support on 4.16 and lower
Fix uprobe support on Ubuntu 18.04.
- Add support for old ld.so.cache format
This fix uprobe support on Debian 10. (and possibly others)
This implement support to parse the original libc5 format.
- Make probe event_alias unique
This fixes issues when trying to attach the same kernel function multiple times on 4.17 and lower (possibly upper too?)
- Do not create data maps on kernel without global data support
Fix map creation failure when a BPF have a data section on older
kernel. (< 5.2)
If the BPF uses that section, relocation will fail accordingly and
report an error.
- Move program's functions to the same map
- update bitflags requirement from 1.2.1 to 2.2.1
Updates the requirements on [bitflags](https://github.com/bitflags/bitflags) to permit the latest version.
- [Release notes](https://github.com/bitflags/bitflags/releases)
- [Changelog](https://github.com/bitflags/bitflags/blob/main/CHANGELOG.md)
- [Commits](https://github.com/bitflags/bitflags/compare/1.2.1...2.2.1)
- update object requirement from 0.30 to 0.31
Updates the requirements on [object](https://github.com/gimli-rs/object) to permit the latest version.
- [Release notes](https://github.com/gimli-rs/object/releases)
- [Changelog](https://github.com/gimli-rs/object/blob/master/CHANGELOG.md)
- [Commits](https://github.com/gimli-rs/object/compare/0.30.0...0.31.0)
---
updated-dependencies:
- dependency-name: object
dependency-type: direct:production
...
- flip feature "no_std" to feature "std"
This fixes `cargo build --all-features` by sidestepping the feature
unification problem described in The Cargo Book[0].
Add `cargo hack --feature-powerset` to CI to enforce that this doesn't
regress (and that all combinations of features work).
Since error_in_core is nightly-only, use core-error and a fake std
module to allow aya-obj to build without std on stable.
[0] https://doc.rust-lang.org/cargo/reference/features.html#feature-unification
- Correctly set the kernel code version for Debian kernel
Fix BPF syscall failure related to the kernel code version.
- Correctly set the kernel code version for Ubuntu kernel
Fix BPF syscall failure related to the kernel code version.
- Add sanitize code for kernels without bpf_probe_read_kernel
Required for kernel before 5.5.
Also move Features to aya-obj.
- Do not use unwrap with btf_fd in bpf_create_map
Fixes a crash when trying to create a map of type BPF_MAP_TYPE_PERCPU_ARRAY when btf_fd is None.
Tested on Ubuntu 18.04 (4.15.0-202-generic)
- support relocations across multiple text sections + fixes
Fix R_BPF_64_64 text relocations in sections other than .text (for
instance .text.unlikely). Also fix misc bugs triggered by integration
tests.
- make it possible to use set_global() with slices of Pod(s)
- make it possible to use set_global() with slices of Pod(s)
- Allow to attach XDP probe by interface index
- Fix MapData Clone implementation
The Clone implementation of MapData was previously not storing the
result of the dup operation.
- Add loaded_programs() API to list all loaded programs
This uses a Programs iterator to yield all loaded bpf programs using
bpf_prog_get_next_id.
- MapData should be Borrow, not AsRef
We don't ever do ref-to-ref conversion for MapData so Borrow should
suffice.
- Fix is_perf_link_supported
This was mistakenly comparing the exit code of the syscall, which is
always -1 and not the corresponding error-code. Added unit tests to
ensure we don't regress.
- More discrete feature logging
Just use the Debug formatter vs. printing a message for each probe.
- Enable bpf_link for perf_attach programs
This adds support for bpf_link to PerfEvent, Tracepoint, Kprobe and
Uprobe programs.
- Add probe for bpf_link_create for perf programs
- Make features a lazy_static
- Add support for multibuffer programs
This adds support for loading XDP programs that are multi-buffer
capable, which is signalled using the xdp.frags section name. When this
is set, we should set the BPF_F_XDP_HAS_FRAGS flag when loading the
program into the kernel.
- Add from_pin for Programs
This commit adds from_pin() which allows the creation of a Program
from a path on bpffs. This is useful to be able to call `attach` or
other APIs for programs that are already loaded to the kernel.
This differs from #444 since it implements this on the concrete program
type, not the Program enum, allowing the user to pass in any additional
context that isn't available from bpf_prog_info.
- fix Lru and LruPerCpu hash maps
They were broken by https://github.com/aya-rs/aya/pull/397
- update documentation and versioning info
- Set the version number of `aya-obj` to `0.1.0`.
- Update the description of the `aya-obj` crate.
- Add a section in README and rustdoc warning about the unstable API.
- add basic documentation to public members
Types relevant to maps are moved into aya_obj_nightly::maps.
Some members are marked `pub(crate)` again.
- migrate aya_nightly::obj into a separate crate
To split the crate into two, several changes were made:
1. Most `pub(crate)` are now `pub` to allow access from Aya;
2. Parts of BpfError are merged into, for example, RelocationError;
3. BTF part of Features is moved into the new crate;
4. `#![deny(missing_docs)]` is removed temporarily;
5. Some other code gets moved into the new crate, mainly:
- aya_nightly::{bpf_map_def, BtfMapDef, PinningType},
- aya_nightly::programs::{CgroupSock*AttachType},
The new crate is currenly allowing missing_docs. Member visibility
will be adjusted later to minimize exposure of implementation details.
- migrate bindgen destination
- make btf::RelocationError private
- fix regression computing pointer sizes
Computing pointer sizes was broken in #285
- fix detaching links on drop
- add missing TryFrom