数据集预处理，划分为测试数据集合验证数据集

在一个数据集中随机选出80%的记录作为训练数据集，训练得到相应的数学模型之后，将剩余的20%的记录作为验证，测试模型的准确性。

原有数据集形式：

1::1193::5::978300760

1::661::3::978302109

1::914::3::978301968

1::3408::4::978300275

1::2355::5::978824291

1::1197::3::978302268

1::1287::5::978302039

1::2804::5::978300719

1::594::4::978302268

1::919::4::978301368

…

…

…

其中[0]-[3]分别代表不同的特征

处理后得到的数据集

3418 869 4 967398834

1545 1073 3 974744490

2851 2572 5 972506781

2068 1198 3 974658346

1031 3633 4 974999133

5555 1073 3 959550355

587 3566 1 975946900

1125 3301 2 988697984

4091 1772 1 965425356

2096 3114 5 974653564

5888 1220 5 957480240

710 93 2 975570015

…

…

…

python代码如下：

import random

def replaced(filename):
new_content=[];
readfile=open(filename)
lines=readfile.readlines()
for line in lines:

new_line=line.strip().replace("::","\t")
'''
contents=line.split("::")
new_line=contents[0].strip()+"\t"+contents[1].strip()+"\t"+contents[2].strip()+"\t"+contents[3].strip()
'''
new_content.append(new_line)
#print contents
#print new_content
readfile.close()
writefile=open(filename.strip(".csv")+"_replaced.csv",'wb')
for i in range(len(new_content)):
writefile.write(new_content[i]+'\n')
writefile.close()

def choose (filename):
content_test=[]
content_result=[]
readfile=open(filename)
lines=readfile.readlines()
sum_length=len(lines)
count=sum_length
print("The numbers of items in "+filename+" is "+str(sum_length))

while count > 0:

if len(content_test)<0.8*sum_length:
i=random.randint(0,count-1)
content_test.append(lines[i])
del lines[i]

count=count-1
for j in range(len(lines)):
content_result.append(lines[j])
readfile.close()

writefile=open("content_test.csv",'wb')
for i in range(len(content_test)):
writefile.write(content_test[i])

writefile.close()

writefile=open("content_result.csv",'wb')
for i in range(len(content_result)):
writefile.write(content_result[i])

writefile.close()

filename="ratings.csv"
choose(filename)

filename_01="content_test.csv"
filename_02="content_result.csv"

replaced(filename_01)
replaced(filename_02)