I have database which each row contains list of numbers, The list of numbers represent a line (The numbers are the change from the last number and not absolute)
I would like to find a way to represent each row with one value, so When I'll have another list of numbers(means new line but not from the DB) I'll be able to order all the rows(which represent lines) from the most similar to the most different in fast way.
- Comparing each row to the list which I have is not an option because I need it to be super fast.
One way I thought of was to calculate the line as a vector length and take all the closest matches of the length from the total list and so to reduce the total list lines to compare, but then I'll still will have to compare each matching length line to the original one, because even if it's the same length it can contains the numbers that assemble the line in different order...
Example of line:
0.56,0.74,1.76,1.49,1.15,0.96,0.95,0.56,0.54,-0.61,-0.33,-0.32,-1.82,-1.72,-2.52,-1.73,-0.56,-0.74,-1.71,0.43,-1.00,-2.77,-2.70,-3.05,-2.98,-2.86,-3.56,-2.58,-0.46,-2.05,-0.53
The problem you are trying to solve is called "nearest neighbor search", and the short answer is, it's a hard problem. I suggest you read up on it – this Wikipedia entry is a good starting point – and then report back to us on what you have found. You could also have a look at this discussion on stackoverflow and the "linked" and "related" questions there, and this discussion at MathWorks, and this one at CodeReview, and probably a lot of others that you can find by typing relevant phrases into the internet.