Reactive MongoDB + Redis
概述
在响应式编程范式下,Spring WebFlux 搭配 MongoDB 和 Redis 是目前构建高吞吐、低延迟数据管道的常见组合。MongoDB 的响应式驱动天然支持非阻塞游标和变更流,Redis 的响应式客户端(Lettuce)基于 Netty 实现事件驱动,两者与 WebFlux 的调度模型完美契合。
本文将深入讲解:
- Spring Data MongoDB Reactive:
ReactiveMongoTemplate与ReactiveMongoRepository ReactiveRedisTemplate的用法与序列化配置- 响应式 Repository 接口设计模式
- MongoDB 事务与 Redis 事务的差异对比
- 实战:社交 Feed 流的响应式缓存 + 数据库组合查询(拉模式 vs 推模式)
1. 环境依赖
1.1 Maven 依赖
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-webflux</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-mongodb-reactive</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-redis-reactive</artifactId>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-pool2</artifactId>
</dependency>1.2 YAML 配置
spring:
data:
mongodb:
uri: mongodb://localhost:27017/social-feed
redis:
host: localhost
port: 6379
timeout: 2s
lettuce:
pool:
max-active: 16
max-idle: 8
min-idle: 22. Spring Data MongoDB Reactive
Spring Data MongoDB Reactive 提供了两套操作入口:Template API 和 Repository API。两者底层共享同一个 ReactiveMongoDatabaseFactory,可以在同一个服务中混合使用。
2.1 ReactiveMongoTemplate
ReactiveMongoTemplate 是响应式 MongoDB 操作的核心类,所有方法返回 Mono<T> 或 Flux<T>。
@Service
public class PostReactiveService {
private final ReactiveMongoTemplate template;
public PostReactiveService(ReactiveMongoTemplate template) {
this.template = template;
}
// 插入文档
public Mono<Post> createPost(Post post) {
return template.insert(post);
}
// 条件查询 + 排序 + 分页
public Flux<Post> findPostsByUserId(String userId, int page, int size) {
Query query = Query.query(Criteria.where("userId").is(userId))
.with(Sort.by(Direction.DESC, "createdAt"))
.skip((long) page * size)
.limit(size);
return template.find(query, Post.class);
}
// 聚合管道
public Flux<PostAggregationResult> aggregateByTag() {
Aggregation aggregation = Aggregation.newAggregation(
Aggregation.group("tag").count().as("count"),
Aggregation.sort(Sort.by(Direction.DESC, "count")),
Aggregation.limit(10));
return template.aggregate(aggregation, "post", PostAggregationResult.class);
}
// 更新
public Mono<UpdateResult> updateContent(String id, String newContent) {
return template.updateFirst(
Query.query(Criteria.where("id").is(id)),
Update.update("content", newContent), Post.class);
}
// 删除
public Mono<DeleteResult> deleteById(String id) {
return template.remove(Query.query(Criteria.where("id").is(id)), Post.class);
}
}2.2 文档映射
@Document(collection = "post")
public class Post {
@Id
private String id;
private String userId;
private String content;
private List<String> tags;
@Field("created_at")
private Instant createdAt;
@Field("updated_at")
private Instant updatedAt;
// getters / setters
}2.3 ReactiveMongoRepository
ReactiveMongoRepository 继承自 ReactiveSortingRepository,提供开箱即用的 CRUD 和方法命名查询推导。
public interface PostRepository extends ReactiveMongoRepository<Post, String> {
Flux<Post> findByUserIdOrderByCreatedAtDesc(String userId);
Flux<Post> findByUserId(String userId, Pageable pageable);
Mono<Long> countByUserId(String userId);
Mono<Void> deleteByUserId(String userId);
}@Service
public class PostRepositoryService {
private final PostRepository repository;
public PostRepositoryService(PostRepository repository) {
this.repository = repository;
}
public Flux<Post> getRecentPosts(String userId) {
return repository.findByUserIdOrderByCreatedAtDesc(userId).take(20);
}
}2.4 Template 与 Repository 的选择
| 维度 | ReactiveMongoTemplate | ReactiveMongoRepository |
|---|---|---|
| 灵活性 | 高:聚合、全文检索、批量更新 | 中:方法命名推导为主 |
| 编码量 | 较多 | 极少 |
| 动态查询 | 强(Query + Criteria) | 弱(需配合 @Query) |
| 事务支持 | 手动控制 Session | 通过 @Transactional |
实践建议:简单 CRUD 用 Repository,复杂聚合用 Template。
3. ReactiveRedisTemplate
3.1 配置与序列化
@Configuration
public class ReactiveRedisConfig {
@Bean
public ReactiveRedisTemplate<String, Object> reactiveRedisTemplate(
ReactiveRedisConnectionFactory factory) {
Jackson2JsonRedisSerializer<Object> jsonSerializer =
new Jackson2JsonRedisSerializer<>(Object.class);
ObjectMapper om = new ObjectMapper()
.registerModule(new JavaTimeModule())
.disable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS);
jsonSerializer.setObjectMapper(om);
StringRedisSerializer keySerializer = StringRedisSerializer.UTF_8;
RedisSerializationContext<String, Object> context =
RedisSerializationContext.<String, Object>newSerializationContext(keySerializer)
.value(jsonSerializer)
.hashKey(keySerializer)
.hashValue(jsonSerializer)
.build();
return new ReactiveRedisTemplate<>(factory, context);
}
}3.2 常用操作
@Service
public class PostCacheService {
private static final String KEY_PREFIX = "post:";
private static final Duration TTL = Duration.ofMinutes(30);
private final ReactiveRedisTemplate<String, Post> redisTemplate;
public PostCacheService(ReactiveRedisTemplate<String, Post> redisTemplate) {
this.redisTemplate = redisTemplate;
}
// 写入 / 读取缓存
public Mono<Boolean> cachePost(Post post) {
return redisTemplate.opsForValue().set(KEY_PREFIX + post.getId(), post, TTL);
}
public Mono<Post> getCachedPost(String postId) {
return redisTemplate.opsForValue().get(KEY_PREFIX + postId);
}
// 批量获取(MGET)
public Flux<Post> getCachedPosts(List<String> postIds) {
List<String> keys = postIds.stream()
.map(id -> KEY_PREFIX + id).collect(Collectors.toList());
return redisTemplate.opsForValue().multiGet(keys)
.flatMapMany(Flux::fromIterable)
.filter(Objects::nonNull);
}
// 列表操作 — Feed 流载体
public Mono<Long> pushToFeed(String feedKey, PostSummary summary) {
return redisTemplate.opsForList().leftPush(feedKey, summary);
}
public Flux<PostSummary> getFeedRange(String feedKey, long start, long end) {
return redisTemplate.opsForList().range(feedKey, start, end);
}
}3.3 响应式事务
ReactiveRedisTemplate 通过 multi() 和 exec() 配合 SessionCallback 实现:
public Mono<List<Object>> executeTransaction(String key1, String key2) {
return redisTemplate.execute(new SessionCallback<List<Object>>() {
@Override
@SuppressWarnings("unchecked")
public <K, V> Mono<List<Object>> execute(ReactiveRedisOperations<K, V> ops) {
return ops.multi()
.flatMap(v -> ops.opsForValue().set((K) key1, (V) "value1"))
.flatMap(v -> ops.opsForValue().set((K) key2, (V) "value2"))
.flatMap(v -> ops.exec());
}
});
}4. 响应式 Repository 接口设计
4.1 Cache-Aside 缓存穿透防护
@Component
public class ReactiveCacheAside<T, ID> {
private final ReactiveRedisTemplate<String, T> cache;
private final ReactiveMongoTemplate db;
private final Class<T> entityClass;
private final String keyPrefix;
public ReactiveCacheAside(ReactiveRedisTemplate<String, T> cache,
ReactiveMongoTemplate db,
Class<T> entityClass,
String keyPrefix) {
this.cache = cache;
this.db = db;
this.entityClass = entityClass;
this.keyPrefix = keyPrefix;
}
public Mono<T> findWithCache(ID id) {
String key = keyPrefix + id;
return cache.opsForValue().get(key)
.switchIfEmpty(Mono.defer(() ->
db.findById(id, entityClass)
.flatMap(entity -> cache.opsForValue()
.set(key, entity, Duration.ofMinutes(30))
.thenReturn(entity))
));
}
public Mono<Void> evictCache(ID id) {
return cache.delete(keyPrefix + id).then();
}
}4.2 批量缓存加载
public Flux<Post> batchLoad(List<String> postIds) {
List<String> cacheKeys = postIds.stream()
.map(id -> "post:" + id).collect(Collectors.toList());
return redisTemplate.opsForValue().multiGet(cacheKeys)
.flatMapMany(Flux::fromIterable)
.filter(Objects::nonNull)
.collectList()
.flatMapMany(cached -> {
Set<String> cachedIds = cached.stream()
.map(Post::getId).collect(Collectors.toSet());
List<String> missingIds = postIds.stream()
.filter(id -> !cachedIds.contains(id))
.collect(Collectors.toList());
if (missingIds.isEmpty()) {
return Flux.fromIterable(cached);
}
return db.find(Query.query(Criteria.where("id").in(missingIds)), Post.class)
.collectList()
.flatMapMany(missing -> {
missing.forEach(post ->
redisTemplate.opsForValue()
.set("post:" + post.getId(), post, Duration.ofMinutes(30))
.subscribe());
return Flux.concat(Flux.fromIterable(cached), Flux.fromIterable(missing));
});
});
}5. 事务处理差异
5.1 对比总览
| 维度 | MongoDB 事务 | Redis 事务 | 传统关系型 DB |
|---|---|---|---|
| ACID 支持 | 完全 ACID(副本集 4.0+) | 仅一致性(无回滚) | 完全 ACID |
| 隔离级别 | 快照隔离(snapshot) | 无隔离(WATCH 乐观锁) | READ COMMITTED 等 |
| 回滚能力 | 支持自动回滚 | 不支持(失败继续执行) | 支持自动回滚 |
| 响应式支持 | inTransaction() | multi() / exec() | R2DBC |
| 适用场景 | 跨文档一致性 | 原子命令序列 | 强一致性业务 |
5.2 MongoDB 响应式事务
@Service
public class PostTransactionService {
private final ReactiveMongoTemplate template;
public PostTransactionService(ReactiveMongoTemplate template) {
this.template = template;
}
public Mono<Post> createPostWithTransaction(Post post, UserAction action) {
return template.inTransaction()
.execute(actions -> template.insert(post)
.flatMap(p -> template.insert(action)
.then(Mono.just(p))))
.next();
}
}5.3 Redis 事务 — 乐观锁示例
public Mono<Boolean> likePostWithRedisTx(String postId, String userId) {
String likeKey = "like:" + postId;
String userSetKey = "liked_users:" + postId;
return redisTemplate.execute(new SessionCallback<List<Object>>() {
@Override
@SuppressWarnings("unchecked")
public <K, V> Mono<List<Object>> execute(ReactiveRedisOperations<K, V> ops) {
return ops.watch((K) likeKey)
.flatMap(v -> ops.opsForValue().get((K) likeKey))
.flatMap(val -> {
int count = val != null ? Integer.parseInt(val.toString()) : 0;
return ops.multi()
.flatMap(v -> ops.opsForValue()
.set((K) likeKey, (V) String.valueOf(count + 1)))
.flatMap(v -> ops.opsForSet().add((K) userSetKey, (V) userId))
.flatMap(v -> ops.exec());
})
.map(results -> !results.isEmpty())
.onErrorReturn(false);
}
});
}5.4 分布式事务的务实选择
跨 MongoDB 和 Redis 的分布式事务代价高昂,通常的折中方案:
- 最终一致性:事件驱动 + 补偿机制
- TCC:缓存层实现 Try 阶段预留资源
- 本地消息表:MongoDB 存事件 → 异步消费后回填 Redis
// 补偿模式:缓存失败时回滚 DB
public Mono<Void> createPostWithCompensation(Post post) {
return db.insert(post)
.flatMap(saved -> cache.cachePost(saved)
.flatMap(cached -> Boolean.TRUE.equals(cached)
? Mono.empty()
: db.remove(Query.query(Criteria.where("id").is(saved.getId())),
Post.class).then()))
.then();
}6. 实战:社交 Feed 流响应式缓存 + 数据库组合查询
6.1 业务场景
构建微博/推特风格的社交平台 Feed 流,要求:
- 展示关注用户的近期帖子
- 延迟 < 200ms
- 支持分页滚动
- 读写比例约 100:1
6.2 Feed 流架构设计思路
推模式(Fan-out Write)
发帖时将帖子推送到所有粉丝的收件箱列表,读操作直接读取收件箱。
- 优点:读延迟极低 O(1)
- 缺点:大 V 发帖时写放大严重
拉模式(Fan-out Read)
发帖只写入作者时间线,读操作实时拉取所有关注者的帖子后合并排序。
- 优点:写入轻量、易于实现
- 缺点:读延迟随关注数线性增长
混合模式(Hybrid)
- 普通用户(粉丝数 < 阈值,如 1 万):推模式
- 大 V 用户(粉丝数 > 阈值):拉模式,大 V 帖子由读取时拉取
6.3 数据结构
// 帖子摘要 — 缓存对象
public class PostSummary {
private String id;
private String userId;
private String userName;
private String contentPreview; // 前 100 字
private Instant createdAt;
private int likeCount;
private int commentCount;
// getters / setters
}
// 收件箱 Redis 存储设计
// Key: feed:inbox:{userId} → List<PostSummary> 保存 500 条,TTL 7 天
// Key: post:{postId} → Post(完整帖子缓存,TTL 30 分钟)6.4 推模式实现
@Service
public class PushBasedFeedService {
private static final int FEED_SIZE = 500;
private static final Duration FEED_TTL = Duration.ofDays(7);
private final ReactiveRedisTemplate<String, PostSummary> redisTemplate;
private final PostRepository postRepository;
private final FollowRepository followRepository;
// 发帖 → 推送给所有粉丝
public Mono<Void> publishPost(Post post) {
return postRepository.save(post)
.flatMapMany(saved -> followRepository
.findFollowerIdsByUserId(saved.getUserId())
.flatMap(followerId -> {
String inboxKey = "feed:inbox:" + followerId;
PostSummary summary = toSummary(saved);
return redisTemplate.opsForList().leftPush(inboxKey, summary)
.then(redisTemplate.opsForList()
.trim(inboxKey, 0, FEED_SIZE - 1))
.then(redisTemplate.expire(inboxKey, FEED_TTL));
}))
.then();
}
// 读取 Feed 流 — O(1) 直接读收件箱
public Flux<PostSummary> getFeed(String userId, int page, int size) {
String inboxKey = "feed:inbox:" + userId;
long start = (long) page * size;
return redisTemplate.opsForList().range(inboxKey, start, start + size - 1);
}
private PostSummary toSummary(Post post) {
PostSummary s = new PostSummary();
s.setId(post.getId());
s.setUserId(post.getUserId());
s.setContentPreview(post.getContent().length() > 100
? post.getContent().substring(0, 100) + "..."
: post.getContent());
s.setCreatedAt(post.getCreatedAt());
s.setLikeCount(post.getLikeCount());
s.setCommentCount(post.getCommentCount());
return s;
}
}6.5 拉模式实现
@Service
public class PullBasedFeedService {
private final PostRepository postRepository;
private final FollowRepository followRepository;
// 读取时实时拉取所有关注者的帖子
public Flux<Post> getFeed(String userId, Instant cursor, int size) {
return followRepository.findFolloweeIdsByUserId(userId)
.collectList()
.flatMapMany(followeeIds -> {
if (followeeIds.isEmpty()) return Flux.empty();
return postRepository
.findByUserIdInAndCreatedAtBeforeOrderByCreatedAtDesc(
followeeIds, cursor, PageRequest.of(0, size));
});
}
}6.6 混合模式完整实现
@Service
public class HybridFeedService {
private static final long VIP_THRESHOLD = 10_000;
private final PushBasedFeedService pushService;
private final PullBasedFeedService pullService;
private final PostRepository postRepository;
private final FollowRepository followRepository;
private final ReactiveRedisTemplate<String, PostSummary> redisTemplate;
// 发帖 — 根据粉丝数决定策略
public Mono<Void> publishPost(Post post) {
return followRepository.countFollowersByUserId(post.getUserId())
.flatMap(count -> {
if (count < VIP_THRESHOLD) {
return pushService.publishPost(post);
}
return postRepository.save(post).then();
});
}
// 读 Feed — 推模式收件箱为主,不足时补充拉取大 V 帖子
public Flux<PostSummary> getFeed(String userId, int page, int size) {
String inboxKey = "feed:inbox:" + userId;
long start = (long) page * size;
return redisTemplate.opsForList().range(inboxKey, start, start + size - 1)
.collectList()
.flatMapMany(cached -> {
if (cached.size() < size) {
int remaining = size - cached.size();
return pullService.getFeed(userId, Instant.now(), remaining)
.map(this::toSummary);
}
return Flux.fromIterable(cached);
});
}
private PostSummary toSummary(Post post) {
PostSummary s = new PostSummary();
s.setId(post.getId());
s.setUserId(post.getUserId());
s.setContentPreview(post.getContent().length() > 100
? post.getContent().substring(0, 100) + "..."
: post.getContent());
s.setCreatedAt(post.getCreatedAt());
return s;
}
}6.7 缓存击穿防护 — 互斥锁
public Mono<Post> getPostWithMutex(String postId) {
String cacheKey = "post:" + postId;
String lockKey = "lock:post:" + postId;
return redisTemplate.opsForValue().get(cacheKey)
.switchIfEmpty(Mono.defer(() ->
redisTemplate.opsForValue()
.setIfAbsent(lockKey, "locked", Duration.ofSeconds(5))
.flatMap(locked -> {
if (Boolean.TRUE.equals(locked)) {
return db.findById(postId, Post.class)
.flatMap(post -> redisTemplate.opsForValue()
.set(cacheKey, post, Duration.ofMinutes(30))
.thenReturn(post))
.finallyDo(() -> redisTemplate.delete(lockKey).subscribe());
}
return Mono.delay(Duration.ofMillis(100))
.flatMap(v -> getPostWithMutex(postId));
})
));
}6.8 冷启动预热
@Component
public class FeedWarmUpRunner implements CommandLineRunner {
private final ReactiveMongoTemplate template;
private final ReactiveRedisTemplate<String, PostSummary> redisTemplate;
@Override
public void run(String... args) {
template.find(Query.query(Criteria.where("createdAt")
.gte(Instant.now().minus(3, ChronoUnit.DAYS)))
.with(Sort.by(Direction.DESC, "likeCount"))
.limit(10_000), Post.class)
.buffer(100)
.flatMap(posts -> Flux.fromIterable(posts)
.groupBy(Post::getUserId)
.flatMap(group -> {
String inboxKey = "feed:inbox:" + group.key();
return group.map(this::toSummary)
.flatMap(s -> redisTemplate.opsForList()
.rightPush(inboxKey, s));
}))
.subscribe();
}
private PostSummary toSummary(Post post) {
return new PostSummary();
}
}6.9 游标分页
Feed 流应使用游标分页而非偏移分页。Redis List 只能通过 index 分页,如需游标分页可结合 ZSet:
public Flux<PostSummary> getFeedByCursor(String userId, double cursor, int size) {
String zsetKey = "feed:zset:" + userId;
return redisTemplate.opsForZSet()
.reverseRangeByScore(zsetKey, cursor, -Double.MAX_VALUE, 0, size)
.flatMap(s -> redisTemplate.opsForValue().get("post:" + s.getId()));
}7. 最佳实践
7.1 线程模型
- 不在响应式链中调用
block() - 耗时计算使用
Schedulers.boundedElastic()隔离
7.2 连接池配置
spring:
data:
redis:
lettuce:
pool:
max-active: 32
max-idle: 16
min-idle: 4
spring:
data:
mongodb:
reactive:
max-connection-pool-size: 32
max-wait-time: 2s7.3 数据一致性策略
| 场景 | 策略 | 实现方式 |
|---|---|---|
| 缓存可丢失 | Cache-Aside | 读 DB 回填,写 DB 后删除缓存 |
| 缓存需强一致 | Write-Through | 同步写 DB + Redis,失败回滚 |
| 秒杀/计数 | Redis + 异步落库 | 先写 Redis,消息队列同步到 MongoDB |
| Feed 流 | 最终一致 | 推模式 + TTL 自动过期 |
8. 结语
Spring WebFlux + Reactive MongoDB + Reactive Redis 是构建高吞吐响应式数据层的成熟方案。核心收益:
- 非阻塞 I/O — 单线程可处理数万并发连接
- 背压支持 — 下游消费能力不足时自动反压上游
- 弹性伸缩 — 无状态 + 响应式驱动天然适配弹性扩缩
Feed 流架构中,推模式与拉模式的选择本质是写放大与读放大的权衡。推模式适合高频读场景,拉模式适合高频写场景,混合模式是工业级系统的务实选择。结合实际场景还应关注 Change Streams 实时增量同步等进阶能力。