Project Description¶

The goal of this project is to provide the ability to take information published by Moodle, in the form of an RSS feed, parse (process) that information, and post new information to a twitter feed.

We need several modules to be able to implement this service. The two most important ones are feedparser and twitter.

feedparser is a library to download an RSS feed parse it into recognizable parts. You just give it the URL for an RSS feed, and it gives you back the entries in the RSS feed in a way that your program can understand them.

twitter is a package that allows you to access information from Twitter, and even to post information to a user's Twitter timeline. To get more information about this package, you can use the command pydoc twitter.Twitter from a terminal window, after installing the package using pip (see documentation at the bottom of this notebook).

import feedparser
import twitter
import os.path
import time

feedURL = 'https://courses201601.hope.edu/rss/file.php/31015/2541d5f2272b1ce53855626a577dbff4/mod_forum/500/rss.xml'

Each of the individual posts to the news forum is contained within one of the components of the entries array

entries=feedparser.parse(feedURL)['entries']

First, we determine when the last time news message that we have seen, but looking for a file named log.txt. If that file doesn't exist, then we use the value None to indicate that we have never seen a message before.

logFilePath = 'log.txt'

lastUpdate = None

if os.path.exists(logFilePath):
    with open(logFilePath) as logfile:
        lastUpdate = time.strptime(logfile.readline().rstrip(), "%a, %d %b %Y %H:%M:%S GMT")

print "Last update date is {0}".format(lastUpdate)

Last update date is time.struct_time(tm_year=2015, tm_mon=12, tm_mday=28, tm_hour=21, tm_min=35, tm_sec=28, tm_wday=0, tm_yday=362, tm_isdst=-1)

The information below is used to connect to Twitter; see the documentation at the bottom of this notebook for information about how to obtain them.

CONSUMER_KEY='72WIHUEbiwAadt8lLQ5K2MZYg'
CONSUMER_SECRET='iap0FBrsrgnz0cwAHnbz4bdQin0As8rYppzUZUYcymYSF1SPa3'
OAUTH_TOKEN='4643042121-6P9V4eV1zuVbaNihZVLBC4y0ql6WCGU5RQ6M8AE'
OAUTH_TOKEN_SECRET='UG0kqSyGJLNDduvh3p1wii6lOxwLCdZoM77hpqGMAj5K0'

Get an authorization token to communicate with Twitter, and then connect to twitter to get access to an object that allows us to interact with the Twitter API

auth = twitter.oauth.OAuth(OAUTH_TOKEN, OAUTH_TOKEN_SECRET, CONSUMER_KEY, CONSUMER_SECRET)
twitter_api = twitter.Twitter(auth = auth)

Now for the "meat" of the program. Go through each of the entries from the RSS feed, determine if the post date is after the last time the program was run, and if so, post an update to the desired twitter account based on the contents of the feed text

tweetsPosted = 0

for entry in entries:
    postDate = time.strptime(entry['published'], "%a, %d %b %Y %H:%M:%S GMT")
    
    if lastUpdate is None or postDate > lastUpdate:
        subject = entry['title_detail']['value']
        link = ' ' + entry['link']
        
        postText = 'News:'
        
        requiredLength = len(postText) + len(link)
        
        availableLength = 140 - requiredLength
        
        ellipsis = ' ... '
        
        if len(subject) > availableLength:
            postText += subject[0:availableLength-len(ellipsis)] + ellipsis
        else:
            postText += subject
        
        postText += link
        
        twitter_api.statuses.update(status = postText)
        tweetsPosted = tweetsPosted + 1
        
print "A total of {0} tweets were posted".format(tweetsPosted)

A total of 0 tweets were posted

Now when we are done we need to update the last updated value stored in the file logfilePath

with open(logFilePath, "w") as logfile:
    logfile.write(entries[0]['published'])

Set up notes¶

Twitter¶

pip install feedparser
pip install twitter

Go to dev.twitter.com/apps and set up a an API key

Click Create New App
Name: CSCI 150 News
Description: Maintain a twitter feed of news posts and other activity on the CSCI 150 Moodle site
Website: http://www.hope.edu/cs/mcfall/
Callback URL: Explicitly left blank

Click Create Twitter Application

Crate OAUTH token by going to the Keys and Access Tokens tab, and then click the button labeled Create my access token.

To view the information available in one of the entries, printed out the contents in a cell, e.g entries[0] This gives a JSON value, unfortunately one that at least the validator at http://codebeautify.org/jsonviewer indicates is invalid. So visually inspecting it indicated that one of the values of interest was the subject, accessible as entries[0]['title_detail']['value']

Facebook¶

This page gives some helpful information about how to get started posting to Facebook.

Page ID: 525981534229454
Register as a Facebook Developer
When asked to add a new app, clicked Advanced, and use Display = Hope Moodle News Forum Poster, Namespace = moodle-news-poster and Category = Education

This gave me an App ID of 1678163185788313 and an App Secret of 4715733e0e8ae7aa37cf95c2c8b249f9.

I set up the Contact Email as mcfall+moodle+facebook@hope.edu

In Step 3 of the tutorial posting, there are options for the type of token when selecting Get Access Token. The value to be used is Get user Access Token. After I tried to get the token, I was prompted to submit to login review; clicking OK through the process seemed to work.

I was given the following short-lived access token:

CAAX2R9CQZAZAkBADWf9HBUXE289jCH5oYmdSgH5mvJnPztqDNZCh6bX2czCVbU5VQyZCq58BCnIHoJb1wbLgTATYQWa90aqLJosu6mZB1sLcPnX5J8BtYZAJJUZBKQXj9tayvQZB0oeEMhdxbkfpD5jl3PwQZCfblT5NU3U1TG1oT6BgeKYULH8WpWseDtjHp60jkDDTsO4ZBtF5cgNN0D8hys

Following the exchange process I get a new, long-lived token:

CAAX2R9CQZAZAkBABs3ZBu97pODgYc3zMIKqXETw0Hcy6MdLXZA1L9y7u1KXySifoZBm3dBSJhXGskgbefkXTI78cqZCejkmvQCBVeBsc8z2UBXpIDJRoTe34BwmRgwTSsNM6RkSDdaaLFdbk73ZCWWzPNnTD3tLUpfMKao81ZAFjadYSKqF9RrZApIz4Wh0LuFRj1gv50WuHnagZDZD

Unfortunately, trying their test script gave the following error:

The user hasn't authorized the application to perform this action

Since Facebook permissions are notoriously annoying, I gave up at this point.

Notes for further development¶

Moodle provides a REST-based API to allow you to get more information; would be nice to be able to post updates when assignments are posted, due dates are changed, etc. It doesn't look possible to do this right at the moment, but it's worth looking into in the future.

This post has information about web services. I tried logging in via Postman but got the following response:

{
  "error": "Web services must be enabled in Advanced features.",
  "stacktrace": null,
  "debuginfo": null,
  "reproductionlink": null
}

Parsing Moodle RSS feeds and posting to social media¶

Project Description¶

Set up notes¶

Twitter¶

Facebook¶

Notes for further development¶