matthias/infomentor
Archived
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

updated changes
All checks were successful
continuous-integration/drone/push Build is passing
Details

Browse Source
This commit is contained in:
Matthias Bilger
2019-09-27 08:45:12 +02:00
parent 999beb6269
commit 92cf128433
14 changed files with 532 additions and 135 deletions
Download Patch File Download Diff File Expand all files Collapse all files

139
.dockerignore Normal file
View File

@@ -0,0 +1,139 @@
infomentor.db
# Created by https://www.gitignore.io/api/python
### Python ###
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
# C extensions
*.so
# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
.hypothesis/
.pytest_cache/
# Translations
*.mo
*.pot
# Django stuff:
*.log
local_settings.py
db.sqlite3
# Flask stuff:
instance/
.webassets-cache
# Scrapy stuff:
.scrapy
# Sphinx documentation
docs/_build/
# PyBuilder
target/
# Jupyter Notebook
.ipynb_checkpoints
# IPython
profile_default/
ipython_config.py
# pyenv
.python-version
# celery beat schedule file
celerybeat-schedule
# SageMath parsed files
*.sage.py
# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/
# Spyder project settings
.spyderproject
.spyproject
# Rope project settings
.ropeproject
# mkdocs documentation
/site
# mypy
.mypy_cache/
.dmypy.json
dmypy.json
### Python Patch ###
.venv/
### Python.VirtualEnv Stack ###
# Virtualenv
# http://iamzed.com/2009/05/07/a-primer-on-virtualenv/
[Bb]in
[Ii]nclude
[Ll]ib
[Ll]ib64
[Ll]ocal
[Ss]cripts
pyvenv.cfg
pip-selfcheck.json
# End of https://www.gitignore.io/api/python
infomentor.ini
infomentor.db
log.*

16
.drone.yml Normal file
View File

@@ -0,0 +1,16 @@
kind: pipeline
type: docker
name: default
steps:
- name: deploy
image: plugins/docker
settings:
registry: registry.d1v3.de
repo: registry.d1v3.de/infomentor
username:
from_secret: docker_username
password:
from_secret: docker_password
tags: ["commit_${DRONE_COMMIT}","build_${DRONE_BUILD_NUMBER}", "latest"]

14
Dockerfile Normal file
View File

@@ -0,0 +1,14 @@
FROM python:3.7.3-stretch
COPY requirements.txt /tmp/
RUN pip install -r /tmp/requirements.txt
COPY . /tmp/infomentor
RUN pip install /tmp/infomentor
RUN useradd --create-home appuser
WORKDIR /home/appuser
USER appuser
VOLUME ["/home/appuser"]
CMD [ "python", "-m", "infomentor" ]

28
README.md
View File

@@ -8,7 +8,7 @@ This tool is designed to check the infomentor portal and send notifications usin
python3 -m venv venv python3 -m venv venv
source venv/bin/activate source venv/bin/activate
python setup.py install python setup.py install
python -m infomentor infomentor
``` ```
After the first run a `infomentor.ini` file is available which has a few values to be entered. After the first run a `infomentor.ini` file is available which has a few values to be entered.
@@ -21,19 +21,19 @@ After the first run a `infomentor.ini` file is available which has a few values
Provide the username and password for infomentor. Provide the username and password for infomentor.
``` ```
source venv/bin/activate source venv/bin/activate
python -m infomentor --adduser <username> adduser --username <username>
``` ```
### Step 2 add notification mechanism ### Step 2 add notification mechanism
``` ```
source venv/bin/activate source venv/bin/activate
python -m infomentor --addmail <username> addmail --username <username>
``` ```
or or
``` ```
source venv/bin/activate source venv/bin/activate
python -m infomentor --addpushover <username> addpushover --username <username>
``` ```
### Step 3 (optional) Add iCloud calendar ### Step 3 (optional) Add iCloud calendar
@@ -42,7 +42,7 @@ It is capable of syncing all the infomentor calendar elements to icloud calendar
``` ```
source venv/bin/activate source venv/bin/activate
python -m infomentor --addcalendar <username> addcalendar --username <username>
``` ```
## NB ## NB
@@ -51,3 +51,21 @@ The login process is a bit scary and mostly hacked. It happens often on the firs
The script shall be run every 10 minutes, that will keep the session alive and minimize errors. The script shall be run every 10 minutes, that will keep the session alive and minimize errors.
## Docker
This could be run within docker. You it has a volume `/home/appuser` where all the data is stored. In favour of accessing it from a webserver you should bindmount it.
There also the infomentor.ini should be placed.
Build the container by `docker build -t infomentor:latest .` and run it like this:
```
docker run -v '/var/docker/infomentor/:/home/appuser' infomentor:latest
```
for adding an user or all the commands run it adding -it to it, like:
```
docker run -it -v '/var/docker/infomentor/:/home/appuser' infomentor:latest adduser
```

2
infomentor.ini.sample
View File

@@ -12,3 +12,5 @@ server = example.org
username = infomentor@example.org username = infomentor@example.org
password = secret1234 password = secret1234
[healthcheck]
url = https://health.d1v3.de/ping/123123123123123

1
infomentor/__init__.py
View File

@@ -0,0 +1 @@
#from infomentor.__main__ import *

105
infomentor/__main__.py
View File

@@ -4,16 +4,17 @@ import argparse
import datetime import datetime
import sys import sys
import os import os
from infomentor import db, model, connector, informer import requests
from infomentor import db, model, connector, informer, config
logformat = "{asctime} - {name:25s} - {levelname:8s} - {message}" logformat = "{asctime} - {name:25s}[{filename:20s}:{lineno:3d}] - {levelname:8s} - {message}"
def logtofile(): def logtofile():
from logging.handlers import RotatingFileHandler from logging.handlers import RotatingFileHandler
handler = RotatingFileHandler("log.txt", maxBytes=51200, backupCount=5) handler = RotatingFileHandler("log.txt", maxBytes=1024*1024, backupCount=10)
logging.basicConfig( logging.basicConfig(
level=logging.INFO, format=logformat, handlers=[handler], style="{" level=logging.INFO, format=logformat, handlers=[handler], style="{"
) )
@@ -28,12 +29,14 @@ def parse_args(arglist):
parser.add_argument( parser.add_argument(
"--nolog", action="store_true", help="print log instead of logging to file" "--nolog", action="store_true", help="print log instead of logging to file"
) )
parser.add_argument("--adduser", type=str, help="add user") parser.add_argument("--adduser", action='store_true', help="add user")
parser.add_argument("--addfake", type=str, help="add fake") parser.add_argument("--addfake", action='store_true', help="add fake")
parser.add_argument("--addpushover", type=str, help="add pushover") parser.add_argument("--addpushover", action='store_true', help="add pushover")
parser.add_argument("--addmail", type=str, help="add mail") parser.add_argument("--addmail", action='store_true', help="add mail")
parser.add_argument("--addcalendar", type=str, help="add icloud calendar") parser.add_argument("--addcalendar", action='store_true', help="add icloud calendar")
parser.add_argument("--addinvitation", action='store_true', help="add calendar invitation")
parser.add_argument("--test", action="store_true", help="test") parser.add_argument("--test", action="store_true", help="test")
parser.add_argument("--username", type=str, nargs='?', help="username")
args = parser.parse_args(arglist) args = parser.parse_args(arglist)
return args return args
@@ -104,6 +107,18 @@ def add_calendar(username):
) )
session.commit() session.commit()
def add_invitation(username):
session = db.get_db()
user = session.query(model.User).filter(model.User.name == username).one_or_none()
if user is None:
print("user does not exist")
return
else:
print(f"Adding Mail for calendar invitation for user: {username}")
mail = input("Mail: ")
user.invitation = model.Invitation(email=mail)
session.commit()
def add_mail(username): def add_mail(username):
session = db.get_db() session = db.get_db()
@@ -123,6 +138,10 @@ def add_mail(username):
def notify_users(): def notify_users():
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
session = db.get_db() session = db.get_db()
cfg = config.load()
if cfg["healthchecks"]["url"] != "":
requests.get(cfg["healthchecks"]["url"])
for user in session.query(model.User): for user in session.query(model.User):
logger.info("==== USER: %s =====", user.name) logger.info("==== USER: %s =====", user.name)
if user.password == "": if user.password == "":
@@ -187,15 +206,17 @@ def main():
logger.info("EXITING - PREVIOUS IS RUNNING") logger.info("EXITING - PREVIOUS IS RUNNING")
raise Exception() raise Exception()
if args.addfake: if args.addfake:
add_fake(args.addfake) add_fake(args.username)
elif args.adduser: elif args.adduser:
add_user(args.adduser) add_user(args.username)
elif args.addpushover: elif args.addpushover:
add_pushover(args.addpushover) add_pushover(args.username)
elif args.addmail: elif args.addmail:
add_mail(args.addmail) add_mail(args.username)
elif args.addcalendar: elif args.addcalendar:
add_calendar(args.addcalendar) add_calendar(args.username)
elif args.addinvitation:
add_invitation(args.username)
else: else:
notify_users() notify_users()
except Exception as e: except Exception as e:
@@ -204,6 +225,64 @@ def main():
finally: finally:
logger.info("EXITING--------------------- %s", os.getpid()) logger.info("EXITING--------------------- %s", os.getpid())
def run_notify():
run_without_args(notify_users)
def run_adduser():
run_with_args(add_user)
def run_addfake():
run_with_args(add_fake)
def run_addpushover():
run_with_args(add_pushover)
def run_addmail():
run_with_args(add_mail)
def run_addcalendar():
run_with_args(add_calendar)
def run_addinvitation():
run_with_args(add_invitation)
def run_with_args(fct):
args = parse_args(sys.argv[1:])
logtofile()
logger = logging.getLogger("Infomentor Notifier")
logger.info("STARTING-------------------- %s", os.getpid())
lock = flock.flock()
try:
if not lock.aquire():
logger.info("EXITING - PREVIOUS IS RUNNING")
raise Exception()
if args.username is None:
print('Provide Username using --username')
raise Exception('No username provided')
fct(args.username)
except Exception as e:
logger.info("Exceptional exit")
logger.exception("Info")
finally:
logger.info("EXITING--------------------- %s", os.getpid())
def run_without_args(fct):
args = parse_args(sys.argv[1:])
logtofile()
logger = logging.getLogger("Infomentor Notifier")
logger.info("STARTING-------------------- %s", os.getpid())
try:
lock = flock.flock()
if not lock.aquire():
logger.info("EXITING - PREVIOUS IS RUNNING")
raise Exception()
fct()
except Exception as e:
logger.info("Exceptional exit")
logger.exception("Info")
finally:
logger.info("EXITING--------------------- %s", os.getpid())
if __name__ == "__main__": if __name__ == "__main__":
main() main()

35
infomentor/config.py
View File

@@ -3,27 +3,36 @@ import os
_config = None _config = None
_defaults = {
"pushover": {
"apikey": "",
},
"general": {
"secretkey": "",
"baseurl": "",
"adminmail": "",
"im1url": "https://im1.infomentor.de/Germany/Germany/Production",
"mimrul": "https://mein.infomentor.de",
},
"smtp": {
"server": "",
"username": "",
"password": "",
},
"healthchecks": {
"url": "",
},
}
def _set_defaults(config): def _set_defaults(config):
config.add_section("pushover") config = _defaults
config.add_section("general")
config.add_section("smtp")
config["pushover"]["apikey"] = ""
config["general"]["secretkey"] = ""
config["general"]["baseurl"] = ""
config["general"]["adminmail"] = ""
config["general"]["im1url"] = "https://im1.infomentor.de/Germany/Germany/Production"
config["general"]["mimurl"] = "https://mein.infomentor.de"
config["smtp"]["server"] = ""
config["smtp"]["username"] = ""
config["smtp"]["password"] = ""
def load(cfg_file="infomentor.ini"): def load(cfg_file="infomentor.ini"):
"""Load the config from the file""" """Load the config from the file"""
global _config global _config
if _config is None: if _config is None:
_config = configparser.ConfigParser() _config = configparser.ConfigParser(_defaults)
if not os.path.isfile(cfg_file): if not os.path.isfile(cfg_file):
_set_defaults(_config) _set_defaults(_config)
save(cfg_file) save(cfg_file)

10
infomentor/connector.py
View File

@@ -10,17 +10,19 @@ import contextlib
import logging import logging
import urllib.parse import urllib.parse
import uuid import uuid
import glob
import hashlib
from infomentor import model, config from infomentor import model, config
class InfomentorFile(object): class InfomentorFile(object):
"""Represent a file which is downloaded""" """Represent a file which is downloaded"""
def __init__(self, directory, filename): def __init__(self, directory, filename, seed=''):
if directory is None: if directory is None:
raise Exception("directory is required") raise Exception("directory is required")
self.filename = filename self.filename = filename
self.randomid = str(uuid.uuid4()) self.randomid = hashlib.sha1('{}{}'.format(filename, seed).encode('utf-8')).hexdigest()
self.directory = directory self.directory = directory
@property @property
@@ -232,7 +234,7 @@ class Infomentor(object):
def _download_file(self, url, directory, filename=None): def _download_file(self, url, directory, filename=None):
"""download a file with provided filename""" """download a file with provided filename"""
file = InfomentorFile(directory, filename) file = InfomentorFile(directory, filename, seed=url)
self.logger.info("to (randomized) directory %s", file.targetdir) self.logger.info("to (randomized) directory %s", file.targetdir)
url = self._mim_url(url) url = self._mim_url(url)
self._do_get(url) self._do_get(url)
@@ -291,7 +293,7 @@ class Infomentor(object):
self.logger.exception("failed to store attachment") self.logger.exception("failed to store attachment")
news = model.News(**storenewsdata) news = model.News(**storenewsdata)
with contextlib.suppress(Exception): with contextlib.suppress(Exception):
news.imagefile = self.get_newsimage(id) news.imagefile = self.get_newsimage(article_json["id"])
return news return news
def get_article(self, id): def get_article(self, id):

53
infomentor/icloudcalendar.py
View File

@@ -2,11 +2,15 @@ from datetime import datetime
import sys import sys
from bs4 import BeautifulSoup from bs4 import BeautifulSoup
import time
import caldav import caldav
from caldav.elements import dav, cdav from caldav.elements import dav, cdav
from lxml import etree from lxml import etree
import requests import requests
from requests.auth import HTTPBasicAuth from requests.auth import HTTPBasicAuth
import logging
_logger = logging.getLogger(__name__)
class iCloudConnector(object): class iCloudConnector(object):
@@ -14,15 +18,9 @@ class iCloudConnector(object):
icloud_url = "https://caldav.icloud.com" icloud_url = "https://caldav.icloud.com"
username = None username = None
password = None password = None
propfind_principal = ( propfind_principal = '<A:propfind xmlns:A="DAV:"><A:prop><A:current-user-principal/></A:prop></A:propfind>'
u"""<?xml version="1.0" encoding="utf-8"?><propfind xmlns='DAV:'>""" propfind_calendar_home_set = "<propfind xmlns='DAV:' xmlns:cd='urn:ietf:params:xml:ns:caldav'><prop> <cd:calendar-home-set/></prop></propfind>"
u"""<prop><current-user-principal/></prop></propfind>"""
)
propfind_calendar_home_set = (
u"""<?xml version="1.0" encoding="utf-8"?><propfind xmlns='DAV:' """
u"""xmlns:cd='urn:ietf:params:xml:ns:caldav'><prop>"""
u"""<cd:calendar-home-set/></prop></propfind>"""
)
def __init__(self, username, password, **kwargs): def __init__(self, username, password, **kwargs):
self.username = username self.username = username
@@ -32,6 +30,7 @@ class iCloudConnector(object):
self.discover() self.discover()
self.get_calendars() self.get_calendars()
# discover: connect to icloud using the provided credentials and discover # discover: connect to icloud using the provided credentials and discover
# #
# 1. The principal URL # 1. The principal URL
@@ -45,40 +44,53 @@ class iCloudConnector(object):
# given credentials # given credentials
headers = {"Depth": "1"} headers = {"Depth": "1"}
auth = HTTPBasicAuth(self.username, self.password) auth = HTTPBasicAuth(self.username, self.password)
principal_response = requests.request( principal_response = self.repeated_request(
"PROPFIND", "PROPFIND",
self.icloud_url, self.icloud_url,
auth=auth, auth=auth,
headers=headers, data=self.propfind_principal
data=self.propfind_principal.encode("utf-8"),
) )
if principal_response.status_code != 207:
print("Failed to retrieve Principal: ", principal_response.status_code)
exit(-1)
# Parse the resulting XML response # Parse the resulting XML response
soup = BeautifulSoup(principal_response.content, "lxml") soup = BeautifulSoup(principal_response.content, "lxml")
self.principal_path = ( self.principal_path = (
soup.find("current-user-principal").find("href").get_text() soup.find("current-user-principal").find("href").get_text()
) )
discovery_url = self.icloud_url + self.principal_path discovery_url = self.icloud_url + self.principal_path
_logger.debug("Discovery url {}".format(discovery_url))
# Next use the discovery URL to get more detailed properties - such as # Next use the discovery URL to get more detailed properties - such as
# the calendar-home-set # the calendar-home-set
home_set_response = requests.request( home_set_response = self.repeated_request(
"PROPFIND", "PROPFIND",
discovery_url, discovery_url,
auth=auth, auth=auth,
headers=headers, data=self.propfind_calendar_home_set,
data=self.propfind_calendar_home_set.encode("utf-8"),
) )
_logger.debug("Result code: {}".format(home_set_response.status_code))
if home_set_response.status_code != 207: if home_set_response.status_code != 207:
print("Failed to retrieve calendar-home-set", home_set_response.status_code) _logger.error("Failed to retrieve calendar-home-set {}".format(home_set_response.status_code))
exit(-1) raise Exception("failed to retrieve calender home set {}".format(home_set_response.content))
# And then extract the calendar-home-set URL # And then extract the calendar-home-set URL
soup = BeautifulSoup(home_set_response.content, "lxml") soup = BeautifulSoup(home_set_response.content, "lxml")
self.calendar_home_set_url = soup.find( self.calendar_home_set_url = soup.find(
"href", attrs={"xmlns": "DAV:"} "href", attrs={"xmlns": "DAV:"}
).get_text() ).get_text()
def repeated_request(self, *args, **kwargs):
for _ in range(0, 5):
response = requests.request(
*args, **kwargs
)
_logger.debug("Request result code: {}".format(response.status_code))
if response.status_code != 207:
_logger.error("Failed to retrieve response: {}".format(response.status_code))
_logger.error("Retry")
time.sleep(0.25)
if response.status_code == 207:
break
else:
raise Exception("failed to retrieve {} {}".format(response.content, response.headers))
return response
# get_calendars # get_calendars
# Having discovered the calendar-home-set url # Having discovered the calendar-home-set url
# we can create a local object to control calendars (thin wrapper around # we can create a local object to control calendars (thin wrapper around
@@ -123,3 +135,4 @@ class iCloudConnector(object):
): ):
# to do # to do
pass pass

196
infomentor/informer.py
View File

@@ -23,13 +23,14 @@ pushover.init(cfg["pushover"]["apikey"])
class Informer(object): class Informer(object):
"""The Logic part of the infomentor notifier. """The Logic part of the infomentor notifier.
This class offers the methods required to notify a user of new News and Homework items posted on infomentor.""" This class offers the methods required to notify a user of new News and Homework items posted on infomentor."""
def __init__(self, user, im, logger): def __init__(self, user, im, logger):
self.logger = logger or logging.getLogger(__name__) self.logger = logger or logging.getLogger(__name__)
self.user = user self.user = user
self.im = im self.im = im
self.cal = None
def send_status_update(self, text): def send_status_update(self, text):
"""In case something unexpected happends and the user has activated the feature to get notified about it, this will send out the information""" """In case something unexpected happends and the user has activated the feature to get notified about it, this will send out the information"""
@@ -56,11 +57,12 @@ class Informer(object):
news = ( news = (
session.query(model.News) session.query(model.News)
.filter(model.News.news_id == news_entry['id']) .filter(model.News.news_id == news_entry['id'])
.filter(model.News.date == news_entry['publishedDate'])
.with_parent(self.user, "news") .with_parent(self.user, "news")
.one_or_none() .one_or_none()
) )
if news is not None: if news is not None:
self.logger.debug('Skipping news') self.logger.debug('Skipping news %s', news_entry['id'])
continue continue
news = self.im.get_news_article(news_entry) news = self.im.get_news_article(news_entry)
self._notify_news(news) self._notify_news(news)
@@ -68,6 +70,9 @@ class Informer(object):
session.commit() session.commit()
def _notify_news(self, news): def _notify_news(self, news):
if self.user.notification is None:
self.logger.debug('Warn: no notification for user')
return
if self.user.notification.ntype == model.Notification.Types.PUSHOVER: if self.user.notification.ntype == model.Notification.Types.PUSHOVER:
self._notify_news_pushover(news) self._notify_news_pushover(news)
elif self.user.notification.ntype == model.Notification.Types.EMAIL: elif self.user.notification.ntype == model.Notification.Types.EMAIL:
@@ -233,77 +238,128 @@ class Informer(object):
s.send_message(mail) s.send_message(mail)
s.quit() s.quit()
def _send_invitation(self, calobj, to, fr="infomentor@09a.de"):
event = calobj.subcomponents[0]
eml_body = event['description']
eml_body_bin = event['description']
msg = MIMEMultipart('mixed')
msg['Reply-To']=fr
msg['Subject'] = event['summary']
msg['From'] = fr
msg['To'] = to
part_email = MIMEText(eml_body,"html")
part_cal = MIMEText(calobj.to_ical().decode('utf-8'),'calendar;method=REQUEST')
msgAlternative = MIMEMultipart('alternative')
msg.attach(msgAlternative)
ical_atch = MIMEBase('application/ics',' ;name="%s"'%("invite.ics"))
ical_atch.set_payload(calobj.to_ical().decode('utf-8'))
encoders.encode_base64(ical_atch)
ical_atch.add_header('Content-Disposition', 'attachment; filename="%s"'%("invite.ics"))
eml_atch = MIMEBase('text/plain','')
eml_atch.set_payload('')
encoders.encode_base64(eml_atch)
eml_atch.add_header('Content-Transfer-Encoding', "")
msgAlternative.attach(part_email)
msgAlternative.attach(part_cal)
self._send_mail(msg)
def _setup_icloudconnector(self):
if self.cal is None:
try:
icx = icloudcalendar.iCloudConnector(
self.user.icalendar.icloud_user, self.user.icalendar.password
)
cname = self.user.icalendar.calendarname
self.cal = icx.get_named_calendar(cname)
if not self.cal:
self.cal = icx.create_calendar(cname)
self.logger.warn("using icloud")
except Exception as e:
self.logger.exception("using icloud dummy connector")
class Dummy(object):
def add_event(self, *args, **kwargs):
pass
self.cal = Dummy()
def _write_icalendar(self, calend):
try:
self._setup_icloudconnector()
self.cal.add_event(calend.to_ical())
except Exception as e:
self.logger.exception('Calendar failed')
def update_calendar(self): def update_calendar(self):
session = db.get_db() session = db.get_db()
if self.user.icalendar is None: if self.user.icalendar is None and self.user.invitation is None:
return return
icx = icloudcalendar.iCloudConnector( try:
self.user.icalendar.icloud_user, self.user.icalendar.password calentries = self.im.get_calendar()
) for entry in calentries:
cname = self.user.icalendar.calendarname self.logger.debug(entry)
cal = icx.get_named_calendar(cname) uid = str(uuid.uuid5(uuid.NAMESPACE_URL, "infomentor_{}".format(entry["id"])))
if not cal: event_details = self.im.get_event(entry["id"])
cal = icx.create_calendar(cname) calend = Calendar()
event = Event()
calentries = self.im.get_calendar() event.add("uid", uid)
for entry in calentries: event.add("summary", entry["title"])
self.logger.debug(entry) event.add("dtstamp", datetime.datetime.now())
uid = "infomentor_{}".format(entry["id"]) if not event_details["allDayEvent"]:
event_details = self.im.get_event(entry["id"]) event.add("dtstart", dateparser.parse(entry["start"]))
self.logger.debug(event_details) event.add("dtend", dateparser.parse(entry["end"]))
calend = Calendar()
event = Event()
event.add("uid", "infomentor_{}".format(entry["id"]))
event.add("summary", entry["title"])
if not event_details["allDayEvent"]:
event.add("dtstart", dateparser.parse(entry["start"]))
event.add("dtend", dateparser.parse(entry["end"]))
else:
event.add("dtstart", dateparser.parse(entry["start"]).date())
event.add("dtend", dateparser.parse(entry["end"]).date())
description = event_details["notes"]
self.logger.debug(event_details['info'])
self.logger.debug(type(event_details['info']))
eventinfo = event_details['info']
self.logger.debug(eventinfo)
self.logger.debug(type(eventinfo))
for res in eventinfo['resources']:
f = self.im.download_file(res["url"], directory="files")
description += """\nAttachment {0}: {2}/{1}""".format(
res['title'], urllib.parse.quote(f), cfg["general"]["baseurl"]
)
event.add("description", description)
calend.add_component(event)
new_cal_entry = calend.to_ical().replace(b"\r", b"")
new_cal_hash = hashlib.sha1(new_cal_entry).hexdigest()
session = db.get_db()
storedata = {
"calendar_id": uid,
"ical": new_cal_entry,
"hash": new_cal_hash,
}
calendarentry = (
session.query(model.CalendarEntry)
.filter(model.CalendarEntry.calendar_id == uid)
.with_parent(self.user, "calendarentries")
.one_or_none()
)
if calendarentry is not None:
if calendarentry.hash == new_cal_hash:
self.logger.info("no change for calendar entry {}".format(uid))
continue
else: else:
self.logger.info("update calendar entry {}".format(uid)) event.add("dtstart", dateparser.parse(entry["start"]).date())
for key, value in storedata.items(): event.add("dtend", dateparser.parse(entry["end"]).date())
setattr(calendarentry, key, value)
else: description = event_details["notes"]
self.logger.info("new calendar entry {}".format(uid)) eventinfo = event_details['info']
calendarentry = model.CalendarEntry(**storedata) new_cal_entry = calend.to_ical().replace(b"\r", b"")
new_cal_hash = hashlib.sha1(new_cal_entry).hexdigest()
for res in eventinfo['resources']:
f = self.im.download_file(res["url"], directory="files")
description += """\nAttachment {0}: {2}/{1}""".format(
res['title'], urllib.parse.quote(f), cfg["general"]["baseurl"]
)
event.add("description", description)
self.user.calendarentries.append(calendarentry) calend.add_component(event)
session.commit() new_cal_entry = calend.to_ical().replace(b"\r", b"")
self.logger.debug(new_cal_entry.decode("utf-8")) session = db.get_db()
cal.add_event(calend.to_ical()) storedata = {
"calendar_id": uid,
"ical": new_cal_entry,
"hash": new_cal_hash,
}
calendarentry = (
session.query(model.CalendarEntry)
.filter(model.CalendarEntry.calendar_id == uid)
.with_parent(self.user, "calendarentries")
.one_or_none()
)
if calendarentry is not None:
if calendarentry.hash == new_cal_hash:
self.logger.info("calendar entry UNCHANGED {}".format(uid))
continue
else:
self.logger.info("calendar entry UPDATED {}".format(uid))
for key, value in storedata.items():
setattr(calendarentry, key, value)
else:
self.logger.info("calendar entry NEW {}".format(uid))
calendarentry = model.CalendarEntry(**storedata)
self.logger.debug(new_cal_entry.decode("utf-8"))
if self.user.icalendar is not None:
self._write_icalendar(calend)
if self.user.invitation is not None:
self._send_invitation(calend, self.user.invitation.email)
self.user.calendarentries.append(calendarentry)
session.commit()
except Exception as e:
self.logger.exception('Calendar failed')

21
infomentor/model.py
View File

@@ -35,6 +35,7 @@ class User(ModelBase):
notification = relationship("Notification", back_populates="user", uselist=False) notification = relationship("Notification", back_populates="user", uselist=False)
apistatus = relationship("ApiStatus", back_populates="user", uselist=False) apistatus = relationship("ApiStatus", back_populates="user", uselist=False)
icalendar = relationship("ICloudCalendar", back_populates="user", uselist=False) icalendar = relationship("ICloudCalendar", back_populates="user", uselist=False)
invitation = relationship("Invitation", back_populates="user", uselist=False)
wantstatus = Column(Boolean) wantstatus = Column(Boolean)
homeworks = relationship("Homework", back_populates="user") homeworks = relationship("Homework", back_populates="user")
news = relationship("News", back_populates="user") news = relationship("News", back_populates="user")
@@ -236,3 +237,23 @@ class ICloudCalendar(ModelBase):
self.icloud_user, self.icloud_user,
self.calendarname, self.calendarname,
) )
class Invitation(ModelBase):
"""An icloud account with a calendar name"""
__tablename__ = "invitation_mails"
id = Column(Integer, primary_key=True)
user_id = Column(Integer, ForeignKey("users.id"))
email = Column(String)
user = relationship("User", back_populates="invitation", uselist=False)
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
def __repr__(self):
return "<Invitation(email='%s')>" % (
self.email,
)

39
requirements.txt
View File

@@ -1,10 +1,29 @@
requests Click==7.0
sqlalchemy Flask==1.1.1
dateparser Flask-Bootstrap==3.3.7.1
python-pushover Jinja2==2.10.1
pycrypto SQLAlchemy==1.3.6
bs4 Werkzeug==0.15.5
flask beautifulsoup4==4.8.0
flask-bootstrap bs4==0.0.1
caldav caldav==0.6.1
icalendar certifi==2019.6.16
chardet==3.0.4
dateparser==0.7.1
dominate==2.3.5
icalendar==4.0.3
idna==2.8
itsdangerous==1.1.0
lxml==4.3.4
pycrypto==2.6.1
python-dateutil==2.8.0
python-pushover==0.4
pytz==2019.1
regex==2019.06.08
requests==2.22.0
six==1.12.0
tzlocal==1.5.1
urllib3==1.25.3
visitor==0.1.3
vobject==0.9.6.1

8
setup.py
View File

@@ -8,6 +8,14 @@ setup(
author_email="matthias@bilger.info", author_email="matthias@bilger.info",
description="grab infomentor news and push or mail them", description="grab infomentor news and push or mail them",
packages=find_packages(), packages=find_packages(),
entry_points = {
'console_scripts': [
'infomentor=infomentor:main',
'adduser=infomentor:run_adduser',
'addmail=infomentor:run_addmail',
'addpushover=infomentor:run_addpushover',
],
},
install_requires=[ install_requires=[
"pycrypto", "pycrypto",
"request", "request",