Duplicate line remover software - 1

Hello, I need a duplicate line remover desktop software. eg: I have this text file - -------- aaa bbb aaa --------- After I run the software, I will get - --------- aaa bbb --------- I have a software to do this alredy, but it's taking too much time and I want a multi-threaded software that can handle large files. I have 100 txt files, each of them is 5 gb, I want to remove duplicate lines in all these files. Software will be multi-threaded and will work very fast, and will use full potential of my ram and cpu processor, I have 2.93 ghz dual quad core and 8 gb ram, I want the software to use all the resources and clean up the file quickly. My server is 64 bit and windows 2003. I have alredy posted a project but reposting it as the guy couldn't make it. I want it done in 1-3 hours, I don't want to waste time. Happy bidding. Thanks