Skip to content

Add in-memory storage provider - #215

Closed
eastagiletracker wants to merge 1 commit into
Centeva:mainfrom
eastagiletracker:agile-board/in-memory-provider
Closed

eastagiletracker wants to merge 1 commit into
Centeva:mainfrom
eastagiletracker:agile-board/in-memory-provider

Conversation

@eastagiletracker

Copy link
Copy Markdown

This PR proposes adding an in-memory storage provider (InMemoryObjectStorage, connection string memory://) so tests can use IObjectStorage without a real filesystem or cloud service. Fixes #7. We include this PR work along with a full history of your repo at https://eastagiletracker.com/projects/734. You can sign in with your GitHub ID to claim ownership of the project.

What changed and why

Issue #7 asks for a provider that is "useful when writing functional or integration tests where you don't want to connect to a real filesystem or cloud service." On current main (ba70e01) there is none: new StorageFactory().CreateConnection("memory://") throws ArgumentException: Could not find a storage provider based on the given connection string (provider: memory).

InMemoryObjectStorage lives in the core package next to DiskObjectStorage and is returned by BuiltinConnectionFactory for memory://, so it works through StorageFactory, AddObjectStorage(config => config.UseConnectionString("memory://")), or new InMemoryObjectStorage() / UseStorage(...). It behaves like the blob providers: a flat keyspace where folders are implied by / (non-recursive listing collapses sub-paths into one folder entry, recursive listing adds the implied folders via FolderHelper), it stores content type and metadata from WriteOptions, implements ISupportsMetadata, and RenameAsync keeps content type and metadata. Written content is copied, and OpenReadAsync returns a read-only stream, so a later write or a caller mutating its buffer never changes what was stored. Every instance is an independent, empty store, and all members are safe to call concurrently. The change is purely additive: no existing type, signature, or default changes, and there are no new package dependencies. The README lists the provider and shows both ways to create it.

Verification

  • InMemoryObjectStorageTests runs the shared CommonObjectStorageTests contract suite (the same one Disk, S3, Blob, Files and GCS run, including the 5,000-object listing test) plus 21 provider-specific tests: content type/metadata round trip, overwrite replacing content and metadata, empty and null streams, read streams isolated from later writes, IncludeMetadata, single subfolder entries, a sibling folder sharing a name prefix not leaking into a listing, delete/rename edge cases, UpdateMetadataAsync (including a missing object throwing FileNotFoundException), canceled tokens, and 200 concurrent writes. All 47 pass.
  • Unit tests cover BuiltinConnectionFactory, StorageFactory, and AddObjectStorage with memory:// (a DI singleton shared with ISupportsMetadata). With the memory:// branch removed from BuiltinConnectionFactory, those 4 tests fail; restored, they pass. With the folder-collapsing condition in ListAsync broken, ListAsync_WithoutRecurseWithFileInFolder_ReturnsFolderOnly and ListAsync_WithoutRecurse_ReturnsEachSubfolderOnce fail.
  • Full dotnet test -c Release (net10.0) before and after: 268 tests before, 319 after, and the failing set is identical (the 129 S3/Azure/GCS integration tests that need cloud credentials I don't have). No new failures. The solution also builds for netstandard2.0 and net472.

If the flat-key RFC in #204 lands, this provider already stores flat keys internally, so moving it to StorageKey/StoragePrefix should be a small change.

How this was managed

This work was tracked as story https://eastagiletracker.com/projects/734/stories/695892 on a board imported from this repo's 213 issues and pull requests, https://eastagiletracker.com/projects/734.

board

If you'd rather not receive contributions like this, reply no-more-prs on this pull request and we won't open any further ones on your repositories.


Lawrence W. Sinclair
CEO / East Agile
linkedin.com/in/lwsinclair/
eastagile.com

InMemoryObjectStorage keeps objects in process memory so functional and
integration tests can use IObjectStorage without a real filesystem or
cloud service. It is available from the built-in connection factory as
"memory://" and runs the shared CommonObjectStorageTests suite.
@pdeffendol

Copy link
Copy Markdown
Contributor

no-more-prs

@pdeffendol pdeffendol closed this Sep 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Create in-memory provider

2 participants