Skip to content

apify push: the zip upload stores symlinks as links and may read the zip before it's written #1548

Closed

Description

@DaveHanns

Problem

When an Actor's files are over the multi-file size limit, apify push packs them into a zip with createActZip and uploads that. This path has two bugs:

  1. A symlinked file goes into the zip as a link. archiver writes a symlink as a link entry, which holds only the target path. The platform unzips it as a link to a file the upload may not hold, for example a schema shared from outside the Actor folder. The multi-file upload sends the file's content, so the same Actor builds or fails depending on its size.
  2. Push may read the zip before it's fully written. createActZip resolves when archive.finalize() does. That happens when the archive stream ends, before the file stream has written the last bytes to disk. Push reads the file right after, so it can upload a truncated zip.

Every push that zips takes this path, not only monorepo pushes.

What we want

  • A file reached through a symlink goes into the zip as its content, as in the multi-file upload.
  • createActZip resolves only once the whole zip is on disk.

How it's tested

Unit tests for createActZip, each failing on the current code: a symlinked file comes out of the zip with its content, and a zip read right after the call holds every file.

Links

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

t-buildersIssues owned by the Builders team.

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions