AI agents run on file systems using standard tools to navigate directories and read file paths.
However, the challenge is that there is a lot of enterprise data in object storage systems, particularly Amazon S3. Object stores serve data through API calls, not file paths. Closing that gap has required a separate file system layer along with S3, mirrored data, and sync channels to keep both aligned.
tThe rise of agent AI makes that challenge even more difficult and was impacting Amazon’s own ability to get things done. AWS engineering teams using tools like Kiro and Claude Code kept running into the same problem: Agents defaulted to local file tools, but the data was in S3. The local download worked until the agent context window became compacted and the session state was lost.
Amazon’s answer is S3 Files, which mounts any S3 bucket directly to an agent’s local environment with a single command. The data remains in S3, without the need for migration. At its core, AWS connects its Elastic File System (EFS) technology to S3 to provide full file system semantics, not a workaround. S3 Files is now available in most AWS Regions.
"By making the data in S3 available immediately, as if it were part of the local file system, we found that we had a really big speedup with the ability for things like Kiro and Claude Code to be able to work with that data." Andy Warfield, vice president and distinguished engineer at AWS, told VentureBeat.
The Difference Between File and Object Storage and Why It Matters
S3 was built to provide durability, scalability, and object-level API-based access. Those properties made it the default storage layer for enterprise data. But they also created a fundamental incompatibility with the file-based tools that developers and agents depend on.
"S3 is not a file system and has no file semantics on many fronts," Warfield said. "You can’t do a move, an atomic move of an object, and there are actually no directories in S3."
Previous attempts to close that gap relied on FUSE (Filesystems in USErspace), a software layer that allows developers to mount a custom file system in user space without changing the underlying storage. Tools like AWS’s Mount Point, Google’s gcsfuse, and Microsoft’s blobfuse2 used FUSE-based drivers to make their respective object stores look like a file system.
Warfield pointed out that the problem is that those object stores were not yet file systems. Those drivers faked file behavior by including additional metadata in buckets, which broke the object API view, or rejected file operations that the object store could not support.
S3 Files adopts a completely different architecture. AWS is connecting its EFS (Elastic File System) technology directly to S3, introducing a full native file system layer and maintaining S3 as the system of record. Both the file system API and the S3 object API remain accessible simultaneously with the same data.
How S3 Files Accelerates Agent AI
Before S3 Files, an agent working with object data had to be explicitly instructed to download files before using the tools. That created a session state issue. As agents compacted their context windows, records of what had been downloaded locally were often lost.
"I would be forced to remind the agent that the data was available locally," Warfield said.
Warfield went over the before and after of a common agent task involving log analysis. He explained that a developer was using Kiro or Claude Code to work with log data, in the case of the single object they would need to tell the agent where the log files are located and go download them. Whereas if the logs can be immediately mounted to the local file system, the developer can simply identify that the logs are in a specific path and the agent immediately has access to review them.
For multi-agent pipelines, multiple agents can access the same mounted repository simultaneously. AWS says thousands of compute resources can connect to a single S3 file system at the same time, with aggregate read performance reaching several terabytes per second, figures VentureBeat could not independently verify.
Shared state between agents works using standard file system conventions: shared subdirectories, note files, and project directories that any agent in the process can read and write. Warfield described AWS engineering teams using this pattern internally, with agents recording research notes and task summaries in shared project directories.
For teams creating RAG channels on agents’ shared content, S3 Vectors – released on AWS re:Invent in December 2024: higher layers for similarity search and improved recovery generation with that same data.
What analysts say: This is not just a better FUSE
AWS is positioning S3 Files against FUSE-based file access from Azure Blob NFS and Google Cloud Storage FUSE. For AI workloads, the significant distinction is not primarily performance.
"S3 Files eliminates data mixing between object and file storage, turning S3 into a low-latency shared workspace without copying data." Jeff Vogel, an analyst at Gartner, told VentureBeat. "The file system becomes a view, not another set of data."
With FUSE-based approaches, each agent maintains its own local view of the data. When multiple agents work simultaneously, those views may not be synchronized.
"It eliminates an entire class of failure modes, including unexplained training/inference failures caused by outdated metadata, which are notoriously difficult to debug." Vogel said. "FUSE-based solutions outsource complexity and problems to the user."
The agent-level implications go even further. The architectural argument matters less than what it reveals in practice.
"For agent AI, which thinks in terms of files, paths, and local scripts, this is the missing link." IDC analyst Dave McCarthy told VentureBeat. "It allows an AI agent to treat an exabyte-scale bucket as its own local hard drive, enabling a level of autonomous operational speed that was previously stifled by the API overhead associated with approaches like FUSE."
Beyond agent workflow, McCarthy sees S3 Files as a broader inflection point in how companies use their data.
"The launch of S3 Files is not just S3 with a new interface; is the removal of the final friction point between big data lakes and autonomous AI," said. "By converging file and object access with S3, they are opening the door to more use cases with less rework."
What this means for businesses
For enterprise teams that have been maintaining a separate file system alongside S3 to support file-based applications or agent workloads, that architecture is now unnecessary.
For enterprise teams consolidating AI infrastructure into S3, the practical change is concrete: S3 is no longer the destination for agent production and becomes the environment where agent work occurs.
"All of these API changes you’re seeing on storage teams come from first-hand work and customer experience using agents to work with data." Warfield said. "We’re really especially focused on removing any friction and making those interactions work as well as possible."
