+- Hack Forums (https://hackforums.net)
+-- Forum: Tech (https://hackforums.net/forumdisplay.php?fid=444)
+--- Forum: Coding (https://hackforums.net/forumdisplay.php?fid=151)
+---- Forum: Software Development (https://hackforums.net/forumdisplay.php?fid=118)
+---- Thread: [C#] Implementing a Dictionary-like structure for large sets? (/showthread.php?tid=5451127)
Pages: Pages
1
2
* - Mistermad - Nov 3, 2016
Why not make your own list?
and create a new Add method so that you can check if the value you want to add is already in the list or not?
The only problem I can see is that this will get slow once you have a few items.
Code
class NewList<T> : List<T>
{
public new void Add(T item)
{
//Check whether item is already in the list
}
}* - severos - Nov 3, 2016
(Nov 3, 2016 12:34 AM)Mistermad Wrote: Why not make your own list?
and create a new Add method so that you can check if the value you want to add is already in the list or not?
The only problem I can see is that this will get slow once you have a few items.
Because I don't want access time O(n) structure, imagine that I have to check over 100million objects just to add one... and imagine the time I have to build the 100million objects at first place -think it's O(n^2)-
* - Snoweyes - Nov 3, 2016
(Nov 2, 2016 09:24 PM)severos Wrote: Thanks for the suggestions, I'll look into them and hope to find a solution.
A single object (key value pair in dictionary): the key is string of a name (variable length but usually not larger than 10 characters) and the value is an object of 2 longs and a byte (around 17 bytes of row data, a bit extra for object and other stuff .Net does). Using a Dictionary I'm able to reach a bit less than 12million objects.
@GameOver
Thanks for the explanation, I'll look further into it and might get inspired to something.
I found this article which might help you: http://www.codeproject.com/Tips/837172/Creating-PartitionedDictionary
Its what The-One mentioned, partitioning the dictionary.
This StackOverflow answer explains why you get a memory out of range exception:
http://stackoverflow.com/a/3657189
* - severos - Nov 3, 2016
(Nov 3, 2016 10:34 AM)Snoweyes Wrote: I found this article which might help you: http://www.codeproject.com/Tips/837172/Creating-PartitionedDictionary
Its what The-One mentioned, partitioning the dictionary.
This StackOverflow answer explains why you get a memory out of range exception:
http://stackoverflow.com/a/3657189
Thanks for the link, I'll read it soon.
As for the out of memory exception, I tried running the diagnostic tools and it was reading 1GB of memory before throwing the exception, no idea why tho.
* - Khamul - Nov 3, 2016
I guess you could implement a HashTable with linear probing? Could be good although deletion would be a problem.