Over 100 posts without posting

So using various bits of python code and other tricks that I have been sharing on here I have made a rather productive blog using python to automate. This blog was only started in the new year and it is now on 100 posts, these posts are not completely meaningless (although making them really useful is still a work in progress). The blog can be found here: Crypto New

This uses the following:

• Pandas - with SQL, from HTML etc and iterrows.
• Twitter Scraping 
• SQL Server - writing to and from
• News gathering
• blogging posting 
• Connecting to Google Sheets 
• Sentiment analysis with NLTK
• And more I am probably forgetting. 

But basically I have a few scripts that I run every day and it produces everything I need. Now all I need to do it set up the facility to do a bit more tweeting and posting about it. Well actually there is a lot more to do. This is sort of a pilot project where I am currently scoping things out. Hopefully I can go on to do much more with this. 

Scraping the News

I thought that quite a cool thing to be able to do would be to get some news stories on a given subject and then create a post on these. Thinking about this in a work environment you might want to pull in news on your company or if you are working on something specific then that topic. 

There is a great site which has an api that allows you to do just this and all very easily. It is returned in json format and my attempts to remove duplicates in pandas has continued to fail, but the overall idea works and just needs to be perfected. In my case I am getting the news stories and then using them to create a blog post on blogger. 

The code: