f4: Facebook’s Warm BLOB Storage System

Created: 2021-11-17T22:09:08-06:00

Return to the Index

This card pertains to a resource available on the internet.

A replacement for Haystack that handles data which is not accessed as frequently.

Relies on Reed Solomon(10,4) and XOR coding instead of simply keeping multiple copies of the file around.

Instead of a directory service the blob information is kept in a graph database.

Accessing data goes through the web frontend / CDN which can do an immediate cache, then internally requesets from blob storage (placing in to hot cache on the way.)

Haystack

Storage is similar to haystack

Stores files similarly to haystack with physical and logical volumes.

Volumes can be locked or unlocked. A locked volume may no longer be written to.

Journal files now exist which can register deletes in locked volumes.

Temperatures of data

Hot: actively requested data; ex. front page images

Warm: data which is accessed recently but not hot

Cold: stored in a manner which requires time and special access to retrieve