Repository navigation
Add in-memory storage provider - #215
Closed
eastagiletracker wants to merge 1 commit into
Closed
eastagiletracker wants to merge 1 commit into
eastagiletracker wants to merge 1 commit into
Conversation
InMemoryObjectStorage keeps objects in process memory so functional and integration tests can use IObjectStorage without a real filesystem or cloud service. It is available from the built-in connection factory as "memory://" and runs the shared CommonObjectStorageTests suite.
Contributor
|
no-more-prs |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR proposes adding an in-memory storage provider (
InMemoryObjectStorage, connection stringmemory://) so tests can useIObjectStoragewithout a real filesystem or cloud service. Fixes #7. We include this PR work along with a full history of your repo at https://eastagiletracker.com/projects/734. You can sign in with your GitHub ID to claim ownership of the project.What changed and why
Issue #7 asks for a provider that is "useful when writing functional or integration tests where you don't want to connect to a real filesystem or cloud service." On current
main(ba70e01) there is none:new StorageFactory().CreateConnection("memory://")throwsArgumentException: Could not find a storage provider based on the given connection string (provider: memory).InMemoryObjectStoragelives in the core package next toDiskObjectStorageand is returned byBuiltinConnectionFactoryformemory://, so it works throughStorageFactory,AddObjectStorage(config => config.UseConnectionString("memory://")), ornew InMemoryObjectStorage()/UseStorage(...). It behaves like the blob providers: a flat keyspace where folders are implied by/(non-recursive listing collapses sub-paths into one folder entry, recursive listing adds the implied folders viaFolderHelper), it stores content type and metadata fromWriteOptions, implementsISupportsMetadata, andRenameAsynckeeps content type and metadata. Written content is copied, andOpenReadAsyncreturns a read-only stream, so a later write or a caller mutating its buffer never changes what was stored. Every instance is an independent, empty store, and all members are safe to call concurrently. The change is purely additive: no existing type, signature, or default changes, and there are no new package dependencies. The README lists the provider and shows both ways to create it.Verification
InMemoryObjectStorageTestsruns the sharedCommonObjectStorageTestscontract suite (the same one Disk, S3, Blob, Files and GCS run, including the 5,000-object listing test) plus 21 provider-specific tests: content type/metadata round trip, overwrite replacing content and metadata, empty and null streams, read streams isolated from later writes,IncludeMetadata, single subfolder entries, a sibling folder sharing a name prefix not leaking into a listing, delete/rename edge cases,UpdateMetadataAsync(including a missing object throwingFileNotFoundException), canceled tokens, and 200 concurrent writes. All 47 pass.BuiltinConnectionFactory,StorageFactory, andAddObjectStoragewithmemory://(a DI singleton shared withISupportsMetadata). With thememory://branch removed fromBuiltinConnectionFactory, those 4 tests fail; restored, they pass. With the folder-collapsing condition inListAsyncbroken,ListAsync_WithoutRecurseWithFileInFolder_ReturnsFolderOnlyandListAsync_WithoutRecurse_ReturnsEachSubfolderOncefail.dotnet test -c Release(net10.0) before and after: 268 tests before, 319 after, and the failing set is identical (the 129 S3/Azure/GCS integration tests that need cloud credentials I don't have). No new failures. The solution also builds fornetstandard2.0andnet472.If the flat-key RFC in #204 lands, this provider already stores flat keys internally, so moving it to
StorageKey/StoragePrefixshould be a small change.How this was managed
This work was tracked as story https://eastagiletracker.com/projects/734/stories/695892 on a board imported from this repo's 213 issues and pull requests, https://eastagiletracker.com/projects/734.
If you'd rather not receive contributions like this, reply
no-more-prson this pull request and we won't open any further ones on your repositories.Lawrence W. Sinclair
CEO / East Agile
linkedin.com/in/lwsinclair/
eastagile.com