Documentation
¶
Index ¶
- type FilePool
- func NewBlockDeviceBackedFilePool(blockDevice blockdevice.BlockDevice, sectorAllocator SectorAllocator, ...) FilePool
- func NewFilePoolFromConfiguration(configuration *pb.FilePoolConfiguration) (FilePool, error)
- func NewMetricsFilePool(base FilePool) FilePool
- func NewQuotaEnforcingFilePool(base FilePool, maximumFileCount, maximumTotalSize uint64) FilePool
- type HoleSource
- type SectorAllocator
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type FilePool ¶
type FilePool interface {
NewFile(holeSource HoleSource, size uint64) (filesystem.FileReadWriter, error)
}
FilePool is an allocator for temporary files. Files are created by calling NewFile(). They are automatically removed by calling Close().
File handles returned by NewFile() are not thread-safe. Additional locking needs to be done at higher levels to permit safe concurrent access.
var EmptyFilePool FilePool = emptyFilePool{}
EmptyFilePool is a FilePool that does not permit the creation of new files. It is used as the default FilePool for the root of the worker's FUSE file system to disallow the creation of files not bound to a specific build action.
func NewBlockDeviceBackedFilePool ¶
func NewBlockDeviceBackedFilePool(blockDevice blockdevice.BlockDevice, sectorAllocator SectorAllocator, sectorSizeBytes int) FilePool
NewBlockDeviceBackedFilePool creates a FilePool that stores all temporary file contents directly on a block device. Using a block device tends to be faster than using a directory on a file system, for the reason that no metadata (e.g., a directory hierarchy and inode attributes) needs to be stored.
func NewFilePoolFromConfiguration ¶
func NewFilePoolFromConfiguration(configuration *pb.FilePoolConfiguration) (FilePool, error)
NewFilePoolFromConfiguration constructs a FilePool based on parameters provided in a configuration file.
func NewMetricsFilePool ¶
NewMetricsFilePool creates a decorator for FilePool that exposes Prometheus metrics on how many files are created and closed.
func NewQuotaEnforcingFilePool ¶
NewQuotaEnforcingFilePool creates a FilePool that enforces disk quotas. It limits how many files may be extracted from an underlying FilePool, while also limiting the total size of all files that are extracted. Space is reclaimed by either truncating files or closing them.
type HoleSource ¶
type HoleSource interface {
io.Closer
io.ReaderAt
GetNextRegionOffset(offset int64, regionType filesystem.RegionType) (int64, error)
Truncate(int64) error
}
HoleSource is used by implementations of FilePool to provide the contents of parts of a file to which no data has been written explicitly (i.e., the holes of a file). This can be used to create files that provide copy-on-write behaviour.
Implementations of FileReader are supposed to behave as ordinary FileReaders, with two differences:
ReadAt() never returns io.EOF. Instead, if an attempt is made to read past the end of the file, an infinite stream of null bytes is returned.
Truncate() can be used to remove data at the end of the HoleSource. Subsequent attempts to read data must return null bytes.
var ZeroHoleSource HoleSource = zeroHoleSource{}
ZeroHoleSource is a primitive implementation of HoleSource that returns null bytes at any given offset. This is sufficient for cases where FilePool is only used to create files that are initially empty.
type SectorAllocator ¶
type SectorAllocator interface {
// Allocate a contiguous range of sectors.
//
// Under high utilization, it may not be possible to allocate
// all space contiguously. In that case, this function returns
// fewer sectors than requested. Repeated calls to this function
// are necessary to request the desired amount of space, albeit
// fragmented.
//
// Sector numbers handed out by this function start at one.
// Zero can be used by the user of this interface for special
// purposes (e.g., sparse files).
AllocateContiguous(maximum int) (uint32, int, error)
// Free a contiguous range of sectors. It is invalid to call
// this function with the first sector number being zero.
FreeContiguous(first uint32, count int)
// Free a potentially fragmented list of sectors. Elements with
// value zero are ignored.
FreeList(sectors []uint32)
}
SectorAllocator is used by BlockDeviceBackedFilePool to allocate space on the block device that is needed to store files.
func NewBitmapSectorAllocator ¶
func NewBitmapSectorAllocator(sectorCount uint32) SectorAllocator
NewBitmapSectorAllocator creates a SectorAllocator that stores information on which sectors are allocated in a bitmap. Sectors are allocated by sequentially scanning the bitmap, continuing where previous calls left off.
Due to its simplicity, this allocator would be prone to heavy fragmentation if used for general purpose file systems. For FilePool this is not a problem, because files have short lifetimes. Fragmentation disappears entirely when a worker goes idle, causing the FilePool to go empty.
Source Files