Machine learning for classifying content in python

Closed - This job posting has been filled and work has been completed.
Web, Mobile & Software Dev Scripts & Utilities Posted 3 years ago

Fixed Price

Delivery by August 2, 2012

$50.00

Budget

Details

We need to classify a lot of text information.


Sample of code:

X_train = np.array(["new york is a hell of a town",
                    "new york was originally dutch",
                    "the big apple is great",
                    "new york is also called the big apple",
                    "nyc is nice",
                    "people abbreviate new york city as nyc",
                    "the capital of great britain is london",
                    "london is in the uk",
                    "london is in england",
                    "london is in great britain",
                    "it rains a lot in london",
                    "london hosts the british museum",
                    "new york is great and so is london",
                    "i like london better than new york"])
y_train = [[0],[0],[0],[0],[0],[0],[1],[1],[1],[1],[1],[1],[0,1],[0,1]]
X_test = np.array(['nice day in nyc',
                   'welcome to london',
                   'hello welcome to new york. enjoy it here and london too'])   

classifier = Pipeline([
    ('vectorizer', CountVectorizer(min_n=1,max_n=2)),
    ('tfidf', TfidfTransformer()),
    ('clf', OneVsRestClassifier(LinearSVC()))])
classifier.fit(X_train, y_train)
predicted = classifier.predict(X_test)
for item, labels in zip(X_test, predicted):
    print '%s => %s' % (item, ', '.join(target_names[x] for x in labels))


We are categorizing urls, so the script must download the content of the precategorized url to build the train system.

Categorized urls: http://www.shallalist.de/Downloads/shallalist.tar.gz


About the Client

(3.22) 12 reviews

Spain
Madrid 08:33 AM

38 Jobs Posted
64% Hire Rate, 1 Open Job

$3,174 Total Spent
34 Hires, 3 Active

$9.25/hr Avg Hourly Rate Paid
278 Hours

Member Since Dec 9, 2010