Struct mmtk::util::metadata::side_metadata::SideMetadataSpec

pub struct SideMetadataSpec {
    pub name: &'static str,
    pub is_global: bool,
    pub offset: SideMetadataOffset,
    pub log_num_of_bits: usize,
    pub log_bytes_in_region: usize,
}

This struct stores the specification of a side metadata bit-set. It is used as an input to the (inline) functions provided by the side metadata module.

Each plan or policy which uses a metadata bit-set, needs to create an instance of this struct.

For performance reasons, objects of this struct should be constants.

Fields

name: &'static str

The name for this side metadata.

is_global: bool

Is this side metadata global? Local metadata is used by certain spaces, while global metadata is used by all the spaces.

offset: SideMetadataOffset

The offset for this side metadata.

log_num_of_bits: usize

Number of bits needed per region. E.g. 0 = 1 bit, 1 = 2 bit.

log_bytes_in_region: usize

Number of bytes of the region. E.g. 3 = 8 bytes, 12 = 4096 bytes (page).

Implementations

impl SideMetadataSpec

pub const fn uses_contiguous_side_metadata(&self) -> bool

Is this spec using contiguous side metadata? If not, it uses chunked side metadata.

pub const fn is_absolute_offset(&self) -> bool

Is offset for this spec Address?

pub const fn is_rel_offset(&self) -> bool

If offset for this spec relative? (chunked side metadata for local specs in 32 bits)

pub const fn get_absolute_offset(&self) -> Address

Get the absolute offset for the spec.

pub const fn get_rel_offset(&self) -> usize

Get the relative offset for the spec.

pub const fn upper_bound_offset(&self) -> SideMetadataOffset

Return the upperbound offset for the side metadata. The next side metadata should be laid out at this offset.

pub const fn upper_bound_address_for_contiguous(&self) -> Address

The upper bound address for metadata address computed for this global spec. The computed metadata address should never be larger than this address. Otherwise, we are accessing the metadata that is laid out after this spec. This spec must be a contiguous side metadata spec (which uses address as offset).

pub(crate) fn assert_metadata_mapped(&self, data_addr: Address)

Used only for debugging. This panics if the required metadata is not mapped

fn assert_value_type<T: MetadataValue>(&self, val: Option<T>)

Used only for debugging.

• Assert if the given MetadataValue type matches the spec.
• Assert if the provided value is valid in the spec.

pub(crate) fn is_mapped(&self, data_addr: Address) -> bool

Check with the mmapper to see if side metadata is mapped for the spec for the data address.

pub(crate) fn zero_meta_bits( meta_start_addr: Address, meta_start_bit: u8, meta_end_addr: Address, meta_end_bit: u8 )

This method is used for bulk zeroing side metadata for a data address range.

pub(crate) fn set_meta_bits( meta_start_addr: Address, meta_start_bit: u8, meta_end_addr: Address, meta_end_bit: u8 )

This method is used for bulk setting side metadata for a data address range.

pub(super) fn bulk_update_metadata( &self, start: Address, size: usize, update_meta_bits: &impl Fn(Address, u8, Address, u8) )

This method does bulk update for the given data range. It calculates the metadata bits for the given data range, and invoke the given method to update the metadata bits.

pub fn bzero_metadata(&self, start: Address, size: usize)

Bulk-zero a specific metadata for a memory region. Note that this method is more sophisiticated than a simple memset, especially in the following cases:

• the metadata for the range includes partial bytes (a few bits in the same byte).
• for 32 bits local side metadata, the side metadata is stored in discontiguous chunks, we will have to bulk zero for each chunk’s side metadata.

Arguments

• start: The starting address of a memory region. The side metadata starting from this data address will be zeroed.
• size: The size of the memory region.

pub fn bset_metadata(&self, start: Address, size: usize)

Bulk set a specific metadata for a memory region. Note that this method is more sophisiticated than a simple memset, especially in the following cases:

• the metadata for the range includes partial bytes (a few bits in the same byte).
• for 32 bits local side metadata, the side metadata is stored in discontiguous chunks, we will have to bulk set for each chunk’s side metadata.

Arguments

• start: The starting address of a memory region. The side metadata starting from this data address will be set to all 1s in the bits.
• size: The size of the memory region.

pub fn bcopy_metadata_contiguous( &self, start: Address, size: usize, other: &SideMetadataSpec )

Bulk copy the other side metadata for a memory region to this side metadata.

This function only works for contiguous metadata. Curently all global metadata are contiguous. It also requires the other metadata to have the same number of bits per region and the same region size.

Arguments

• start: The starting address of a memory region.
• size: The size of the memory region.
• other: The other metadata to copy from.

fn side_metadata_access<const CHECK_VALUE: bool, T: MetadataValue, R: Copy, F: FnOnce() -> R, V: FnOnce(R)>( &self, data_addr: Address, input: Option<T>, access_func: F, verify_func: V ) -> R

This is a wrapper method for implementing side metadata access. It does nothing other than calling the access function with no overhead, but in debug builds, it includes multiple checks to make sure the access is sane.

• check whether the given value type matches the number of bits for the side metadata.
• check if the side metadata memory is mapped.
• check if the side metadata content is correct based on a sanity map (only for extreme assertions).

pub unsafe fn load<T: MetadataValue>(&self, data_addr: Address) -> T

Non-atomic load of metadata.

Safety

This is unsafe because:

1. Concurrent access to this operation is undefined behaviour.
2. Interleaving Non-atomic and atomic operations is undefined behaviour.

pub unsafe fn store<T: MetadataValue>(&self, data_addr: Address, metadata: T)

Non-atomic store of metadata.

Safety

This is unsafe because:

1. Concurrent access to this operation is undefined behaviour.
2. Interleaving Non-atomic and atomic operations is undefined behaviour.

pub fn load_atomic<T: MetadataValue>( &self, data_addr: Address, order: Ordering ) -> T

Loads a value from the side metadata for the given address. This method has similar semantics to store in Rust atomics.

pub fn store_atomic<T: MetadataValue>( &self, data_addr: Address, metadata: T, order: Ordering )

Store the given value to the side metadata for the given address. This method has similar semantics to store in Rust atomics.

pub unsafe fn set_zero(&self, data_addr: Address)

Non-atomically store zero to the side metadata for the given address. This method mainly facilitates clearing multiple metadata specs for the same address in a loop.

Safety

This is unsafe because:

1. Concurrent access to this operation is undefined behaviour.
2. Interleaving Non-atomic and atomic operations is undefined behaviour.

pub fn set_zero_atomic(&self, data_addr: Address, order: Ordering)

Atomiccally store zero to the side metadata for the given address. This method mainly facilitates clearing multiple metadata specs for the same address in a loop.

pub unsafe fn set_raw_byte_atomic(&self, data_addr: Address, order: Ordering)

Atomically store one to the side metadata for the data address with the possible side effect of corrupting and setting the entire byte in the side metadata to 0xff. This can only be used for side metadata smaller than a byte. This means it does not only set the side metadata for the data address, and it may also have a side effect of corrupting and setting the side metadata for the adjacent data addresses. This method is only intended to be used as an optimization to skip masking and setting bits in some scenarios where setting adjancent bits to 1 is benign.

Safety

This method may corrupt and set adjacent bits in the side metadata as a side effect. The user must make sure that this behavior is correct and must not rely on the side effect of this method to set bits.

pub unsafe fn load_raw_byte(&self, data_addr: Address) -> u8

Load the raw byte in the side metadata byte that is mapped to the data address.

Safety

This is unsafe because:

1. Concurrent access to this operation is undefined behaviour.
2. Interleaving Non-atomic and atomic operations is undefined behaviour.

pub unsafe fn load_raw_word(&self, data_addr: Address) -> usize

Load the raw word that includes the side metadata byte mapped to the data address.

Safety

This is unsafe because:

1. Concurrent access to this operation is undefined behaviour.
2. Interleaving Non-atomic and atomic operations is undefined behaviour.

pub fn compare_exchange_atomic<T: MetadataValue>( &self, data_addr: Address, old_metadata: T, new_metadata: T, success_order: Ordering, failure_order: Ordering ) -> Result<T, T>

Stores the new value into the side metadata for the gien address if the current value is the same as the old value. This method has similar semantics to compare_exchange in Rust atomics. The return value is a result indicating whether the new value was written and containing the previous value. On success this value is guaranteed to be equal to current.

fn fetch_ops_on_bits<F: Fn(u8) -> u8>( &self, data_addr: Address, meta_addr: Address, set_order: Ordering, fetch_order: Ordering, update: F ) -> u8

This is used to implement fetch_add/sub for bits. For fetch_and/or, we don’t necessarily need this method. We could directly do fetch_and/or on the u8.

pub fn fetch_add_atomic<T: MetadataValue>( &self, data_addr: Address, val: T, order: Ordering ) -> T

Adds the value to the current value for this side metadata for the given address. This method has similar semantics to fetch_add in Rust atomics. Returns the previous value.

pub fn fetch_sub_atomic<T: MetadataValue>( &self, data_addr: Address, val: T, order: Ordering ) -> T

Subtracts the value from the current value for this side metadata for the given address. This method has similar semantics to fetch_sub in Rust atomics. Returns the previous value.

pub fn fetch_and_atomic<T: MetadataValue>( &self, data_addr: Address, val: T, order: Ordering ) -> T

Bitwise ‘and’ the value with the current value for this side metadata for the given address. This method has similar semantics to fetch_and in Rust atomics. Returns the previous value.

pub fn fetch_or_atomic<T: MetadataValue>( &self, data_addr: Address, val: T, order: Ordering ) -> T

Bitwise ‘or’ the value with the current value for this side metadata for the given address. This method has similar semantics to fetch_or in Rust atomics. Returns the previous value.

pub fn fetch_update_atomic<T: MetadataValue, F: FnMut(T) -> Option<T> + Copy>( &self, data_addr: Address, set_order: Ordering, fetch_order: Ordering, f: F ) -> Result<T, T>

Fetches the value for this side metadata for the given address, and applies a function to it that returns an optional new value. This method has similar semantics to fetch_update in Rust atomics. Returns a Result of Ok(previous_value) if the function returned Some(_), else Err(previous_value).

pub unsafe fn find_prev_non_zero_value<T: MetadataValue>( &self, data_addr: Address, search_limit_bytes: usize ) -> Option<Address>

Search for a data address that has a non zero value in the side metadata. The search starts from the given data address (including this address), and iterates backwards for the given bytes (non inclusive) before the data address.

The data_addr and the corresponding side metadata address may not be mapped. Thus when this function checks the given data address, and when it searches back, it needs to check if the address is mapped or not to avoid loading from an unmapped address.

This function returns an address that is aligned to the region of this side metadata (log_bytes_per_region), and the side metadata for the address is non zero.

Safety

This function uses non-atomic load for the side metadata. The user needs to make sure that there is no other thread that is mutating the side metadata.

fn find_prev_non_zero_value_simple<T: MetadataValue>( &self, data_addr: Address, search_limit_bytes: usize ) -> Option<Address>

fn find_prev_non_zero_value_fast<T: MetadataValue>( &self, data_addr: Address, search_limit_bytes: usize ) -> Option<Address>

pub fn scan_non_zero_values<T: MetadataValue>( &self, data_start_addr: Address, data_end_addr: Address, visit_data: &mut impl FnMut(Address) )

Search for data addresses that have non zero values in the side metadata. This method is primarily used for heap traversal by scanning the VO bits.

This function searches the side metadata for the data address range from data_start_addr (inclusive) to data_end_addr (exclusive). The data address range must be fully mapped.

For each data region that has non-zero side metadata, visit_data is called with the lowest address of that region. Note that it may not be the original address used to set the metadata bits.

fn scan_non_zero_values_simple<T: MetadataValue>( &self, data_start_addr: Address, data_end_addr: Address, visit_data: &mut impl FnMut(Address) )

fn scan_non_zero_values_fast( &self, data_start_addr: Address, data_end_addr: Address, visit_data: &mut impl FnMut(Address) )

Trait Implementations

impl Clone for SideMetadataSpec

fn clone(&self) -> SideMetadataSpec

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for SideMetadataSpec

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Hash for SideMetadataSpec

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl PartialEq for SideMetadataSpec

fn eq(&self, other: &SideMetadataSpec) -> bool

This method tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

This method tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Copy for SideMetadataSpec

impl Eq for SideMetadataSpec

impl StructuralPartialEq for SideMetadataSpec