Topics 40

Caching Patterns Cluster Sharding & Hash Slots Clustering Connection Pooling & Client Libraries Data Types Distributed Locks with Redis Expiration & Eviction Geospatial Data in Redis Hashes in Redis HyperLogLog in Redis Keyspace Notifications Leaderboards with Sorted Sets Lists in Redis Lua Scripting Persistence (RDB/AOF) Pub/Sub Rate Limiting with Redis Redis Architecture & Installation Redis as a Message Queue Redis Backup & Disaster Recovery Redis CLI & Basic Commands Redis Memory Optimization Redis Modules Overview Redis Monitoring & Observability Redis Performance Tuning & Benchmarking Redis Replication Redis Security & ACL Redis Sentinel & High Availability Redis vs Memcached Redis with Docker & Kubernetes RedisBloom (Probabilistic Data Structures) RedisJSON RedisSearch (Full Text Search) RedisTimeSeries Session Management with Redis Sets & Sorted Sets in Redis Streams Strings & Bitmaps in Redis TLS & Encryption in Redis Transactions

HyperLogLog in Redis

Q: What is a HyperLogLog in Redis, and what problem does it solve?

A HyperLogLog is a probabilistic data structure used to estimate the number of unique items in a very large collection using a tiny, fixed amount of memory, solving the problem of counting unique visitors or unique events at massive scale without needing to store every single individual item to check for duplicates.

Q: How do you add items to a HyperLogLog and get an estimated count of unique items using Redis?

You use PFADD to add one or more items to a HyperLogLog structure, and PFCOUNT to retrieve the current estimated number of unique items that have been added, with Redis automatically creating the HyperLogLog structure the first time you add an item to a new key.

Q: How accurate is a Redis HyperLogLog's estimated count, and why might a small amount of inaccuracy be an acceptable tradeoff?

A Redis HyperLogLog typically has a standard error of about zero point eight one percent, meaning the estimated count will usually be very close to the true value, and this small inaccuracy is often an entirely acceptable tradeoff in exchange for using a fixed, extremely small amount of memory instead of the potentially massive amount required to store and check every individual unique item exactly.

Q: How do you merge multiple HyperLogLog structures together to get a combined unique count across them?

You use the PFMERGE command, specifying a destination key and the source HyperLogLog keys you want to combine, which creates a new HyperLogLog representing the union of all unique items across the merged structures, letting you calculate combined unique counts, such as total unique visitors across several different days.

Q: How would you design an analytics system using HyperLogLog to track unique visitors across multiple time periods, such as daily, weekly, and monthly counts?

You would create a separate HyperLogLog for each day, add visitor identifiers to the appropriate day's structure as they occur, and then use PFMERGE to combine the relevant daily structures into weekly or monthly HyperLogLogs whenever those aggregated counts are needed, avoiding the need to reprocess raw visitor data for every different time period you want to report on.

Q: What are the limitations of HyperLogLog that make it unsuitable for certain use cases, even though it is excellent for approximate unique counting?

HyperLogLog can only give you the approximate count of unique items and cannot tell you what those specific items actually are, cannot remove a previously added item from the count, and its small error margin makes it unsuitable for situations requiring an exact count, such as billing calculations based on precise usage numbers.

Q: How does the internal sparse and dense representation of a HyperLogLog in Redis affect its memory usage as the number of unique items grows?

Redis initially stores a HyperLogLog using a compact sparse representation when the number of unique items is still small, automatically converting to a denser, slightly larger fixed size representation once the count of unique items grows beyond a certain threshold, balancing memory efficiency for smaller counts against the need for accuracy at larger scales.

7 questions found

What is a HyperLogLog in Redis, and what problem does it solve?

Beginner

A HyperLogLog is a probabilistic data structure used to estimate the number of unique items in a very large collection using a tiny, fixed amount of memory, solving the problem of counting unique visitors or unique events at massive scale without needing to store every single individual item to check for duplicates.

PFADD unique_visitors:today 'user123'
PFADD unique_visitors:today 'user456'
PFCOUNT unique_visitors:today

Real-world example A high traffic website tracks approximately how many unique visitors it has had today using a HyperLogLog, storing this estimate in just a few kilobytes of memory regardless of whether there are thousands or millions of actual unique visitors.

Common follow-ups: How accurate is a HyperLogLog's count compared to an exact count?;How much memory does a HyperLogLog actually use regardless of how many items are added?

Data Types;Redis Memory Optimization

How do you add items to a HyperLogLog and get an estimated count of unique items using Redis?

Beginner

You use PFADD to add one or more items to a HyperLogLog structure, and PFCOUNT to retrieve the current estimated number of unique items that have been added, with Redis automatically creating the HyperLogLog structure the first time you add an item to a new key.

PFADD page_views:homepage 'visitor1' 'visitor2' 'visitor3'
PFCOUNT page_views:homepage

Real-world example A marketing analytics dashboard tracks the estimated number of unique visitors to each page on a website, adding visitor identifiers with PFADD and displaying the running estimate using PFCOUNT.

Common follow-ups: What data type does PFCOUNT actually return?;Does adding the exact same item twice affect the count?

Redis CLI & Basic Commands;Redis Monitoring & Observability

How accurate is a Redis HyperLogLog's estimated count, and why might a small amount of inaccuracy be an acceptable tradeoff?

Intermediate

A Redis HyperLogLog typically has a standard error of about zero point eight one percent, meaning the estimated count will usually be very close to the true value, and this small inaccuracy is often an entirely acceptable tradeoff in exchange for using a fixed, extremely small amount of memory instead of the potentially massive amount required to store and check every individual unique item exactly.

-- A HyperLogLog uses roughly 12 kilobytes
-- regardless of counting a thousand or a billion items
PFADD large_dataset 'item1' 'item2' -- ... millions more

Real-world example An analytics platform accepts a small margin of error in its unique visitor counts in exchange for using a tiny, predictable amount of memory, since knowing the count is approximately ten million rather than exactly ten million is perfectly sufficient for their reporting needs.

Common follow-ups: In what scenarios would this small margin of error actually matter?;How does the error rate change as the number of unique items grows very large?

Data Types;Redis Memory Optimization

How do you merge multiple HyperLogLog structures together to get a combined unique count across them?

Intermediate

You use the PFMERGE command, specifying a destination key and the source HyperLogLog keys you want to combine, which creates a new HyperLogLog representing the union of all unique items across the merged structures, letting you calculate combined unique counts, such as total unique visitors across several different days.

PFMERGE weekly_unique_visitors day1_visitors day2_visitors day3_visitors
PFCOUNT weekly_unique_visitors

Real-world example An analytics system merges seven daily HyperLogLog structures together at the end of each week using PFMERGE, calculating an estimated total of unique visitors across the entire week without needing to reprocess all the original raw data.

Common follow-ups: Does merging HyperLogLogs preserve accuracy as well as counting unique items directly?;Can you merge a HyperLogLog with itself without causing any issues?

Redis Performance Tuning & Benchmarking;Data Types

How would you design an analytics system using HyperLogLog to track unique visitors across multiple time periods, such as daily, weekly, and monthly counts?

Advanced

You would create a separate HyperLogLog for each day, add visitor identifiers to the appropriate day's structure as they occur, and then use PFMERGE to combine the relevant daily structures into weekly or monthly HyperLogLogs whenever those aggregated counts are needed, avoiding the need to reprocess raw visitor data for every different time period you want to report on.

PFADD visitors:2026-09-07 'user123'
PFMERGE visitors:week36 visitors:2026-09-01 visitors:2026-09-02 -- and so on

Real-world example A media company tracks daily unique visitor HyperLogLogs throughout the month, merging them into weekly and monthly rollups on demand, giving their analytics team flexible reporting across multiple time granularities without reprocessing raw visitor logs.

Common follow-ups: How long should individual daily HyperLogLog structures be retained before being cleaned up?;Does this approach introduce any additional error compared to counting directly over the full time period?

Redis Memory Optimization;Redis Backup & Disaster Recovery

What are the limitations of HyperLogLog that make it unsuitable for certain use cases, even though it is excellent for approximate unique counting?

Advanced

HyperLogLog can only give you the approximate count of unique items and cannot tell you what those specific items actually are, cannot remove a previously added item from the count, and its small error margin makes it unsuitable for situations requiring an exact count, such as billing calculations based on precise usage numbers.

-- HyperLogLog cannot list the actual unique items,
-- only estimate how many there are
PFCOUNT unique_visitors -- returns a count, not the actual visitor list

Real-world example A billing system that needs to charge customers based on an exact count of unique API calls avoids using HyperLogLog for this specific calculation, instead using an exact counting method since even a small error could result in an incorrect charge.

Common follow-ups: What exact counting alternatives exist for situations where HyperLogLog's approximation is not acceptable?;Can you ever retrieve the actual list of items that were added to a HyperLogLog?

Sets & Sorted Sets in Redis;Data Types

How does the internal sparse and dense representation of a HyperLogLog in Redis affect its memory usage as the number of unique items grows?

Intermediate

Redis initially stores a HyperLogLog using a compact sparse representation when the number of unique items is still small, automatically converting to a denser, slightly larger fixed size representation once the count of unique items grows beyond a certain threshold, balancing memory efficiency for smaller counts against the need for accuracy at larger scales.

-- Small HyperLogLogs use a more compact sparse encoding
-- automatically converting to dense encoding as they grow
STRLEN unique_visitors:today -- shows current memory footprint

Real-world example A team monitoring memory usage notices a HyperLogLog's storage size grows slightly as more unique visitors are added throughout the day, understanding this is the expected transition from the sparse to dense internal representation.

Common follow-ups: At what point does Redis switch from sparse to dense representation?;Does this internal representation change affect how you interact with the HyperLogLog commands?

Redis Memory Optimization;Data Types