Delivery of fresh press using Python directly to the mailbox
This article will highlight the following features of python:
All this will be accompanied by working examples.
Many people remember the standard movie scene from a typical morning in the American commuter posolka: the guy on a cycling box rides and throws away a newspaper. And then a kind, big, shaggy dog brings a newspaper to the head of the family. But progress is being made and newspapers can now be obtained in digital format, you can even order their delivery by mail. On some sites this opportunity is provided for money, on my favorite, but terribly far from the world of IT Sport Express, this opportunity is absent altogether. But it is possible to download pdf.
In this connection, every morning you had to go to the site, follow the link there, continue to download to your computer, then send to your kindle-box, then turn on wifi on Kindle - and now you can read the newspaper in the subway and electric train
To work with the web, the urllib2 module is used. This module allows you to get the contents of the pages, as well as a lot more. But in this case we have enough of one of its functions: the page view.
')
with is the equivalent of using in C # and try in Java 7. In short, with guarantees that the resources captured in this block will be released as soon as we go beyond its limits (in the case of a normal exit or, for example, an exception).
In order to get a link to the archive with a fresh issue of a newspaper, we use the re module, which implements all the power of regular expressions. However, as in the case of urllib2, we again do not need anything supernatural:
(? P) - this is how the named group is defined in the regular schedule. This way we get the link and file name.
Now that we have content that we plan to send, it would be nice to create an “envelope” for it.
To work with letters in python, the module is used ... the module is used ... the email module is used.
To begin with we will create the simple text letter. Details about MIME are available, of course, here .
It's time to create a function to attach a file. If you want your email client to open attachments, you must specify a file_type.
In order to get the message from the message to send, call the method as_string ().
Now when there is a letter, you need to be able to send it. Smtplib is responsible for sending emails to Python.
Imagine that we have a Message class with the get_from (), get_to (), and get_text () methods.
To maintain 'with' you need to add a couple of methods:
Now we have everything we need.
My final script script looks like this:
After each launch of this script, the latest release of Sport Express goes straight to your inbox! I propose to share in the comments, what other publications distribute free pdf versions of magazines. In principle, it is possible and not pdf, as long as it is possible to receive fresh issues on a regular basis.
And so, in order to download a new magazine - you just need to change 3 functions.
I hope that the material was useful and pleasant for the reader.
- parsing a web page using a simple regular expression;
- downloading a file from a web page;
- sending the downloaded file through an smtp server;
- writing a small generalizing script.
All this will be accompanied by working examples.
Prehistory
Many people remember the standard movie scene from a typical morning in the American commuter posolka: the guy on a cycling box rides and throws away a newspaper. And then a kind, big, shaggy dog brings a newspaper to the head of the family. But progress is being made and newspapers can now be obtained in digital format, you can even order their delivery by mail. On some sites this opportunity is provided for money, on my favorite, but terribly far from the world of IT Sport Express, this opportunity is absent altogether. But it is possible to download pdf.
In this connection, every morning you had to go to the site, follow the link there, continue to download to your computer, then send to your kindle-box, then turn on wifi on Kindle - and now you can read the newspaper in the subway and electric train
Simple web page analysis
To work with the web, the urllib2 module is used. This module allows you to get the contents of the pages, as well as a lot more. But in this case we have enough of one of its functions: the page view.
def download_by_link(self, link): content = urllib2.urlopen(link).read() # html, link link_on_file, filename = self.get_filename(content) # html fullname = self.get_dir_name() + filename if self.is_new_version(fullname): with open(fullname, 'w') as fd: content = urllib2.urlopen(link_on_file).read() # fd.write(content) return self.get_prepared_files(fullname) ')
with is the equivalent of using in C # and try in Java 7. In short, with guarantees that the resources captured in this block will be released as soon as we go beyond its limits (in the case of a normal exit or, for example, an exception).
In order to get a link to the archive with a fresh issue of a newspaper, we use the re module, which implements all the power of regular expressions. However, as in the case of urllib2, we again do not need anything supernatural:
def get_filename(self, text): pattern = r'(?P<link>http:\/\/archive\.sport\-express\.ru\/pdf\/(?P<filename>[0-9]+\.zip))' match = re.search(pattern, text) return match.group('link'), match.group('filename')
(? P) - this is how the named group is defined in the regular schedule. This way we get the link and file name.
class SportExpress(): def get_from(self): return 'username@mail.ru' def get_title(self): return 'Sport Express subscribe' def get_text(self): return "Good morning! Don't be lazy - read newspaper!" def get_prepared_files(self, archive): directory_to_extract = archive[:archive.rfind('.')] zipfile.ZipFile(archive).extractall(directory_to_extract) # res = glob.glob(directory_to_extract+'/*.*') return res def get_dir_name(self): dir_name = 'archive/sport-express/' if not os.path.exists(dir_name): os.mkdir(dir_name) return dir_name def is_new_version(self, filename): return os.path.exists(self.get_dir_name() + filename)
Create a letter
Now that we have content that we plan to send, it would be nice to create an “envelope” for it.
To work with letters in python, the module is used ... the module is used ... the email module is used.
import email, email. encoders, email.mime.text, email.mime.base class MessageBase: def __init__(self, subscriber): self.__to_addresses = subscriber.get_subscribers() self.__from_address = subscriber.get_from() self.__message = self.__create_message(subscriber.get_title(), subscriber.get_text())
To begin with we will create the simple text letter. Details about MIME are available, of course, here .
def __create_message(self, subject, text): email_msg = email.MIMEMultipart.MIMEMultipart('mixed') email_msg['Subject'] = subject email_msg['From'] = self.get_from() email_msg['To'] = ', '.join(self.get_to()) email_msg.attach(email.mime.text.MIMEText(text,'text')) return email_msg
It's time to create a function to attach a file. If you want your email client to open attachments, you must specify a file_type.
def attach_file(self, filename, file_link, file_type = 'unknown'): file_message = email.mime.base.MIMEBase('application', file_type) file_message.set_payload(file(file_link).read()) email.encoders.encode_base64(file_message) file_message.add_header('Content-Disposition','attachment;filename='+filename) self.__message.attach(file_message) return True
In order to get the message from the message to send, call the method as_string ().
def get_text(self): return self.__message.as_string()
We send the letter
Now when there is a letter, you need to be able to send it. Smtplib is responsible for sending emails to Python.
import smtplib from message import MessageBase class SmtpBase: def __init__(self, serverhost): self.__smtp_server = serverhost def open(self): self.__server = smtplib.SMTP(self.__smtp_server) self.__server.login('<enter your smtp login>', '<enter your smtp password>') def close(self): self.__server.quit()
Imagine that we have a Message class with the get_from (), get_to (), and get_text () methods.
def send_mail(self, message): for message_to in message.get_to(): self.__server.sendmail(message.get_from(), message_to, message.get_text())
To maintain 'with' you need to add a couple of methods:
def __enter__(self): self.open() return self def __exit__(self, type, value, traceback): if type: print '%s: %s %s' % (type, value, traceback) self.close()
Generalize
Now we have everything we need.
My final script script looks like this:
from subscriber_sex import SportExpress from smtp import SmtpBase from message import MessageBase b = SportExpress() filenames = b.download_by_link('http://www.sport-express.ru') msg = MessageBase(b) for file in filenames: msg.attach_file(file[file.rfind('/')+1:], file, 'pdf') with SmtpBase('smtp.yandex.ru') as s: s.send_mail(msg)
After each launch of this script, the latest release of Sport Express goes straight to your inbox! I propose to share in the comments, what other publications distribute free pdf versions of magazines. In principle, it is possible and not pdf, as long as it is possible to receive fresh issues on a regular basis.
And so, in order to download a new magazine - you just need to change 3 functions.
I hope that the material was useful and pleasant for the reader.
Source: https://habr.com/ru/post/150940/
All Articles