Target

"Twitter mood predicts the stock market"
https://arxiv.org/pdf/1010.3003.pdf
Its conclusion can be roughly summarized below.

Calmness on twitter correlates with Dow Jones Index Average and it enables us to predict up-down of DJI index on 87.6% accuracy

Validation steps

Extract calmness score from Twitter dataset of 2009

https://archive.org/details/twitter_cikm_2010

Method is almost same with the thesis. But I can't get questionaries called "Profile of Mood States"(POMS). So I selected words express calmness with my original method

Predict up-down of DJI Index from calmness score with Xgboost library

Data&Prediction code: https://gist.github.com/ryogrid/e5274ac31a5b9ac02e8a9fe33b8ac921

Data format: date, daily calmness score, daily DJI Index

Early half part of data for learning. Latter half for prediction

Input data are 3 days calmness score and stock price diff (price[t]-price[t-1])

Plotted calmness score

DJI Index of 2009

Result

I successfully predicted up-down on 59.5% accuracy.
If you doubt of this result, please try prediction with my data or analyze it on statistical view.

https://gist.github.com/ryogrid/e5274ac31a5b9ac02e8a9fe33b8ac921

If you have any question.
Please mail to ryo.contact [at] gmail.com