Ensemble found real bugs in LangChain and Next.js

Verified by follow-up PRs. Click through to check.

Security
langchain-ai/langchain
validate paths in prompt.save and load_prompt #36200
+337 / -37 6 files
libs/core/langchain_core/prompts/loading.py
+def _validate_path(path: Path) -> None:
+    if path.is_absolute():
+        raise ValueError(f"Path '{path}' is absolute...")
+    if ".." in path.parts:
+        raise ValueError(f"Path '{path}' contains '..' ...")
S Security Engineer
  • Path validation checks for .. and absolute paths, but does not resolve symlinks via os.path.realpath(). An attacker can create a symlink pointing outside the allowed directory to bypass both checks.
Confirmed: Fixed in follow-up PR #36585, which added Path.resolve() before validation.
Confidence Score Breakdown
Finding: "Missing symlink resolution in path validation"
Senior Code Reviewer
→
not flagged (focused on code clarity)
Security Engineer
→
Flagged "No realpath() — symlink bypass possible"
Architecture Reviewer
→
Flagged "Path.resolve() should be called first"
2 of 3 reviewers independently flagged this issue
High Confidence
Why this matters
Single agent
"Path validation looks good" → ships with bug
Ensemble
2/3 flagged symlink bypass → blocked before merge

Why 3 reviewers beat 1

Same PR. Same code. Different coverage.

Single Agent
1 One reviewer runs general-purpose analysis
✓ Finds: "Path validation added for .. and absolute paths"
✗ Misses: symlink bypass — no realpath() check
Result
"Looks good to me" ✓ Approved
Confidence: unknown
Ensemble — 3 Reviewers
Sr Senior: "Code structure looks correct" ✓
S Security: "Missing symlink resolution — bypass possible" ⚠
A Architecture: "Validation should use Path.resolve()" ⚠
Result
2/3 flagged an issue → HIGH confidence finding
Confidence: HIGH — 67%
Disagreement = signal
Get this on your PRs — install free

Read-only access · Uninstall in one click

Regression
vercel/next.js
TypeScript plugin: Allow functions in action-like props #67211
+127 lines
A Architecture Reviewer
  • When adding the Action suffix heuristic, the existing exception for the reset prop in error.tsx was removed. This will cause false TypeScript errors for valid code.
Confirmed: Fixed in #69777, which restored the reset prop allowlist.

What you'll see on every PR

Ensemble posts a single comment with findings and confidence scores.

E
ensemble-code-review bot commented 3 weeks ago

Ensemble Review

3 reviewers · 2 findings · Confidence: HIGH

Severity Finding File Reviewers
Warning Path validation missing symlink resolution loading.py:25 Security Architecture
Info Consider using pathlib consistently loading.py:30 Senior

Confidence breakdown

2/3 reviewers flagged: Missing Path.resolve() before validation
1/3 reviewers noted: Style suggestion (non-blocking)
Reviewed by: Senior Code Reviewer ✓ · Security Engineer ✓ · Architecture Reviewer ✓
👍 2 🚀 1

Get this on every PR

Install in 30 seconds. Free for public repos.

Install Ensemble Or try on a single PR
Read-only access · Code deleted after review · Uninstall in one click