test: add e2e tests by l2ysho · Pull Request #1077 · apify/apify-cli

l2ysho · 2026-04-09T22:13:14Z

runCli() helper with auto-detected dist/npx/global modes + 6 help tests
test-actor helper + basic-actor fixture + 8 local tests (invalid-config, actor-run-input)
8 builds tests with auth, platform push, and remote cleanup
e2e.yaml CI workflow — triggers after beta publish, daily, manual dispatch with version override
Deleted cucumber infrastructure

Commands tested:

apify help, actor help, apify help help, apify help -h, apify help run, apify run --help — help output and entrypoints
apify run — with --input (inline JSON), -i (alias), --input-file (from file, missing file, stdin via -), implicit stdin, invalid actor.json
apify builds create — with invalid ID, unpublished actor, published actor, --json
apify builds info — with invalid ID, valid build ID, --json
apify builds ls --json

Replace cucumber's subprocess approach with a simpler runCli() helper that spawns the real built CLI binary. Auto-detects environment via is-ci: uses dist mode locally, npx mode in CI. - Add runCli() helper (test/e2e/__helpers__/run-cli.ts) - Add 6 help command e2e tests (test/e2e/help.test.ts) - Add test:e2e and test:e2e:local scripts - Exclude test/e2e from test:local

Migrate invalid-config and actor-run-input cucumber features to Vitest. Refactor runCli() to accept array args instead of string command parsing. - Add test-actor helper (create, cleanup, corrupt, read results) - Add basic-actor.js fixture - Add invalid-config.test.ts (1 test) - Add actor-run-input.test.ts (7 tests) - Refactor runCli(binary, args) array API — no shell quoting issues

When an exit code assertion fails, the error message now includes stderr output for easier debugging. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Migrate builds-namespace cucumber feature to Vitest. Tests auth via CLI subprocess, push to platform, and exercise builds create/info/ls. - Add builds.test.ts (8 tests, requires TEST_USER_TOKEN) - Use getApifyClientOptions for correct client config - Share build ID via beforeAll in builds info describe block

New workflow runs e2e tests after every beta publish (workflow_run), daily at 06:00 UTC (schedule), and on manual dispatch with optional version override. Temporary pull_request trigger for validation. - Auto-detects CI → npx apify-cli@beta mode - Manual dispatch supports global mode with specific versions - Repo guard prevents fork execution - Command injection protection via env var indirection - 30min timeout, concurrency control - Fix || vs ?? for empty APIFY_CLI_E2E_MODE handling Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

All 22 cucumber scenarios have been migrated to Vitest e2e tests. - Delete /features/ directory (4 feature files, 6 implementation files) - Delete cucumber.json - Delete .github/workflows/cucumber.yaml - Remove @cucumber/cucumber from devDependencies - Remove test:cucumber script - Remove features from lint paths - Update stale cucumber reference in call.test.ts

On a fresh clone, test/tmp is gitignored and doesn't exist, causing e2e tests to fail with ENOENT when execa tries to use it as cwd. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

vladfrangu · 2026-04-13T07:39:18Z

-        "test:local": "vitest run --testNamePattern '^((?!\\[api]).)*$' --exclude ./test/api",
+        "test:local": "vitest run --testNamePattern '^((?!\\[api]).)*$' --exclude ./test/api --exclude ./test/e2e",
+        "test:e2e": "vitest run ./test/e2e",
+        "test:e2e:local": "vitest run ./test/e2e --exclude '**/builds*'",


Lets use the same prefix matcher as current tests please

you mean use [e2e] in --testNamePattern yop ?

vladfrangu · 2026-04-13T07:41:08Z

+            - name: Install dependencies
+              run: yarn
+
+            - name: Install specific CLI version


So we are not testing bundles too? Only npm publishes?

You can technically install bundles with your version then rename the binaries so you can test them alongside npm

good point, lets do separate issue to address bundles to not overcomplicate this PR

Open an issue in advance 🙏

B4nan

As we briefly discussed this yesterday, I'd like to run this by default on a fresh build of the CLI instead of using the published beta version, so we can actually test the code from GH. Otherwise no strong opinions, we gotta start somewhere.

l2ysho · 2026-04-15T13:04:00Z

@B4nan @vladfrangu addressed, should I remove e2e step from PR pipeline before merge or we will wait until it gets too heavy to have it here?

B4nan · 2026-04-15T13:07:17Z

If it can run in a few minutes (or should I say, if its not the slowest job), I'd vote to keep it there.

…r-apify-cli' into 1065-add-end-to-end-e2e-tests-for-apify-cli

l2ysho and others added 6 commits April 9, 2026 16:44

fix(test): add stderr context to e2e exit code assertions

7846d43

When an exit code assertion fails, the error message now includes stderr output for easier debugging. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

l2ysho linked an issue Apr 9, 2026 that may be closed by this pull request

Add end-to-end (e2e) tests for Apify CLI #1065

Open

2 tasks

github-actions bot assigned l2ysho Apr 9, 2026

github-actions bot added this to the 138th sprint - Tooling team milestone Apr 9, 2026

github-actions bot added t-tooling Issues with this label are in the ownership of the tooling team. tested Temporary label used only programatically for some analytics. labels Apr 9, 2026

l2ysho and others added 3 commits April 10, 2026 00:34

fix(test): ensure test/tmp directory exists before creating test actors

76460c6

On a fresh clone, test/tmp is gitignored and doesn't exist, causing e2e tests to fail with ENOENT when execa tries to use it as cwd. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

format:fix

a346d38

fix(ci): remove windows from e2e matrix, fix empty env var handling

e999265

l2ysho marked this pull request as ready for review April 10, 2026 07:44

l2ysho requested a review from vladfrangu as a code owner April 10, 2026 07:44

l2ysho requested a review from B4nan April 10, 2026 07:45

vladfrangu requested changes Apr 13, 2026

View reviewed changes

fix path

4c17c33

l2ysho requested a review from vladfrangu April 13, 2026 09:48

B4nan changed the title ~~1065 add end to end e2e tests for apify cli~~ test: add e2e tests Apr 14, 2026

B4nan reviewed Apr 14, 2026

View reviewed changes

l2ysho added 3 commits April 15, 2026 14:32

update name pattern

14a02aa

simplify and remove modes

98eefac

Merge branch 'master' into 1065-add-end-to-end-e2e-tests-for-apify-cli

1e475b4

l2ysho mentioned this pull request Apr 15, 2026

Add bundle-mode coverage to e2e tests #1081

Open

l2ysho added 2 commits April 15, 2026 15:39

fix test matcher

6f1a8c3

Merge remote-tracking branch 'origin/1065-add-end-to-end-e2e-tests-fo…

c1ca04c

…r-apify-cli' into 1065-add-end-to-end-e2e-tests-for-apify-cli

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test: add e2e tests#1077

test: add e2e tests#1077
l2ysho wants to merge 15 commits intomasterfrom
1065-add-end-to-end-e2e-tests-for-apify-cli

l2ysho commented Apr 9, 2026 •

edited

Loading

Uh oh!

vladfrangu Apr 13, 2026

Uh oh!

l2ysho Apr 13, 2026 •

edited

Loading

Uh oh!

vladfrangu Apr 14, 2026

Uh oh!

vladfrangu Apr 13, 2026

Uh oh!

l2ysho Apr 13, 2026

Uh oh!

vladfrangu Apr 14, 2026

Uh oh!

Uh oh!

B4nan left a comment

Uh oh!

l2ysho commented Apr 15, 2026

Uh oh!

B4nan commented Apr 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

l2ysho commented Apr 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vladfrangu Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

l2ysho Apr 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vladfrangu Apr 14, 2026

Choose a reason for hiding this comment

Uh oh!

vladfrangu Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

l2ysho Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

vladfrangu Apr 14, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

B4nan left a comment

Choose a reason for hiding this comment

Uh oh!

l2ysho commented Apr 15, 2026

Uh oh!

B4nan commented Apr 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

l2ysho commented Apr 9, 2026 •

edited

Loading

l2ysho Apr 13, 2026 •

edited

Loading