Skip to content

chore(website): migrate to pnpm and enable minimum release age#1845

Open
B4nan wants to merge 7 commits intomasterfrom
chore/migrate-website-to-pnpm
Open

chore(website): migrate to pnpm and enable minimum release age#1845
B4nan wants to merge 7 commits intomasterfrom
chore/migrate-website-to-pnpm

Conversation

@B4nan
Copy link
Copy Markdown
Member

@B4nan B4nan commented Apr 14, 2026

Summary

Part of the org-wide supply-chain hardening + pnpm migration. Migrates the website/ subdirectory of crawlee-python from Yarn 4 to pnpm. Only website/ changes — the Python root (uv) is untouched. Adds @apify/* + @crawlee/* whitelist at the Renovate layer (minimumReleaseAge: "1 day" was already set).

Changes

  • website/package.json: renamed to crawlee-python-website, marked private: true; packageManager: pnpm@10.24.0; yarn / npxpnpm in scripts
  • website/pnpm-workspace.yaml: minimumReleaseAge: 1440 with @apify/* + @crawlee/* excluded
  • website/.npmrc: node-linker=hoisted, link-workspace-packages=true, prefer-workspace-packages=true, public-hoist-pattern[]=*
  • website/docusaurus.config.js: rename future.experimental_fasterfuture.faster (Docusaurus 3.10 renamed the key; pnpm resolves 3.10 from the ^3.9.2 range)
  • Deleted website/yarn.lock and website/.yarnrc.yml; generated website/pnpm-lock.yaml
  • pyproject.toml poe tasks build-docs / run-docs: drop corepack enable && yarn && yarn build/start, use pnpm install --frozen-lockfile && pnpm build/start
  • New .github/actions/pnpm-install composite action (cached pnpm store, pattern from chore: move to pnpm from yarn apify-cli#1068)
  • manual_release_stable.yaml version_docs job: use pnpm/action-setup + pnpm install --frozen-lockfile instead of corepack enable && yarn install; npx docusauruspnpm exec docusaurus; also commits pnpm-lock.yaml alongside versioned docs
  • renovate.json: add internalChecksFilter: "strict" and @apify/* + @crawlee/* whitelist packageRule (minimumReleaseAge: "1 day" was already present)

🤖 Generated with Claude Code

Migrates the website/ subdirectory from Yarn 4 to pnpm. The Python
root (uv) is untouched. Adds @apify/* + @crawlee/* whitelist at the
Renovate layer (minimumReleaseAge was already present in renovate.json).

- website/package.json: rename to crawlee-python-website + private: true,
  packageManager -> pnpm@10.24.0, yarn/npx -> pnpm
- website/pnpm-workspace.yaml: minimumReleaseAge 1440 with @apify/* +
  @crawlee/* excluded
- website/.npmrc: hoisted linker + workspace linking flags
- website/docusaurus.config.js: rename future.experimental_faster ->
  future.faster (Docusaurus 3.10 renamed the key)
- Delete website/yarn.lock and website/.yarnrc.yml
- pyproject.toml poe tasks build-docs/run-docs: drop corepack + yarn,
  use pnpm install --frozen-lockfile + pnpm build/start
- New .github/actions/pnpm-install composite action (pattern from
  apify/apify-cli#1068)
- manual_release_stable.yaml version_docs job: pnpm/action-setup +
  pnpm install --frozen-lockfile instead of corepack + yarn;
  npx docusaurus -> pnpm exec docusaurus; commit pnpm-lock.yaml
- renovate.json: add internalChecksFilter "strict" and @apify/* +
  @crawlee/* whitelist (minimumReleaseAge "1 day" was already set)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@github-actions github-actions bot added this to the 138th sprint - Tooling team milestone Apr 14, 2026
@github-actions github-actions bot added the t-tooling Issues with this label are in the ownership of the tooling team. label Apr 14, 2026
@B4nan B4nan added the adhoc Ad-hoc unplanned task added during the sprint. label Apr 14, 2026
@codecov
Copy link
Copy Markdown

codecov bot commented Apr 14, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.32%. Comparing base (f241ead) to head (49a31eb).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1845      +/-   ##
==========================================
- Coverage   92.34%   92.32%   -0.02%     
==========================================
  Files         157      157              
  Lines       10921    10921              
==========================================
- Hits        10085    10083       -2     
- Misses        836      838       +2     
Flag Coverage Δ
unit 92.32% <ø> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@B4nan B4nan marked this pull request as ready for review April 15, 2026 13:41
Replaces the local .github/actions/pnpm-install composite copy
with the shared one from apify/workflows@main. Identical behavior,
less duplication.
Move working-directory to job-level defaults so the composite
action's 'pnpm install' runs in website/. Override explicitly for
the Python uv step.
@Pijukatel Pijukatel self-requested a review April 15, 2026 14:02
Copy link
Copy Markdown
Collaborator

@Pijukatel Pijukatel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait, we need to add uv specific excludes newer, I will add soon

Copy link
Copy Markdown
Collaborator

@Pijukatel Pijukatel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add this to pyproject.toml

[tool.uv.exclude-newer-package]
apify-client = false 
apify-shared = false 
apify_fingerprint_datapoints = false

apify-client, apify-shared, and apify_fingerprint_datapoints are
internal; their releases should install immediately (no 24h wait).
Mirrors the @apify/* + @crawlee/* whitelist in pnpm-workspace.yaml
on the JS side.
@B4nan
Copy link
Copy Markdown
Member Author

B4nan commented Apr 15, 2026

Added in commit b4f9b59 — plus the same block in apify-client-python and apify-sdk-python. On the JS side, apify-client, apify, crawlee, and got-scraping were added to minimumReleaseAgeExclude in pnpm-workspace.yaml across the 11 JS repos.

B4nan added 2 commits April 15, 2026 18:46
Silences npm warnings about unknown options like node-linker; pnpm
reads the same keys from pnpm-workspace.yaml in camelCase form.
Block accidental npm/yarn install — npm 10.5+ and pnpm 10.x both
honor devEngines.packageManager and refuse to run when it doesn't
match.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

adhoc Ad-hoc unplanned task added during the sprint. t-tooling Issues with this label are in the ownership of the tooling team.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants