[Mistermad]

Why not make your own list?
and create a new Add method so that you can check if the value you want to add is already in the list or not?
The only problem I can see is that this will get slow once you have a few items.

Code

class NewList<T> : List<T>
{
  public new void Add(T item)
  {
    //Check whether item is already in the list
  }
}

[severos]

Why not make your own list?
and create a new Add method so that you can check if the value you want to add is already in the list or not?
The only problem I can see is that this will get slow once you have a few items.

Because I don't want access time O(n) structure, imagine that I have to check over 100million objects just to add one... and imagine the time I have to build the 100million objects at first place -think it's O(n^2)-

[Snoweyes]

Thanks for the suggestions, I'll look into them and hope to find a solution.
A single object (key value pair in dictionary): the key is string of a name (variable length but usually not larger than 10 characters) and the value is an object of 2 longs and a byte (around 17 bytes of row data, a bit extra for object and other stuff .Net does). Using a Dictionary I'm able to reach a bit less than 12million objects.

@GameOver
Thanks for the explanation, I'll look further into it and might get inspired to something.

I found this article which might help you: http://www.codeproject.com/Tips/837172/C...Dictionary
Its what The-One mentioned, partitioning the dictionary.

This StackOverflow answer explains why you get a memory out of range exception:
http://stackoverflow.com/a/3657189

[severos]

I found this article which might help you: http://www.codeproject.com/Tips/837172/C...Dictionary
Its what The-One mentioned, partitioning the dictionary.

This StackOverflow answer explains why you get a memory out of range exception:
http://stackoverflow.com/a/3657189

Thanks for the link, I'll read it soon.

As for the out of memory exception, I tried running the diagnostic tools and it was reading 1GB of memory before throwing the exception, no idea why tho.

[Khamul]

I guess you could implement a HashTable with linear probing? Could be good although deletion would be a problem.