Python packaging is infamous and it is hard to use. Thankfully, a lot of progress has been made in recent years. Piping, PyPI, virtualenv, and wheels.
I'd say to talk to the advanced audience. Ideally, it will be a clear idea of ​​how to optimize. :)
As a penniless academic I wanted to do “big data” for science. Open source, python, and simple patterns were the way forward. Staying on top of todays datasets is an arm race. Data analytics machinery — clusters, NOSQL, visualization, Hadoop, machine learning, ... —can be a resource thin. It makes it easy to find out how to use it. For the out-of-the-core and parallel computing.
Identity of data processing with Python. What are the emerging patterns in data processing? How can modern data-mining ideas be used? What constraints and design trade-offs govern projects like scikit-learn, Mayavi, or joblib? Simple framework-less code?
This report propose Python-learning method with Minecraft. Minecraft is a popular sandbox game in the world. We are using Mincraft and Mod (Extending Minecraft Functionality) for Python learning. Mod is using RaspberryJam to provide Python to Minecraft. You can use minecraft in Python by using it.
Often, when developing, we are faced with situations where, to increase the productivity of an application, it is necessary to do large refactorings, to change the architecture. But what to do if there is no time for this and you need performance right here and now? Of course, first of all, you need to resort to traditional medicine - to profile and correct the bottlenecks. But much has already been said about this. I would like to talk about several unconventional techniques that once helped me overcome performance problems.
Frontera allows you to build a distributed robot for downloading pages from the Internet in large volumes in real time. It can also be used to build focused robots to siphon a subset of previously known websites. In addition to the description of the framework and system requirements, I will talk about our experience of downloading Spanish Internet using Fronter and present some statistics.
In the report: Web-client. Primitive request. Use authorization and sessions. Web server Primitive example. Complex routes. Web sockets. Client and server. Processing parallel tasks (asyncio.wait) Testing. Explicit loop Databases. Redis, Postgres, Mongo. Sessions Authorization and access rights. Debugtoolbar. * PEP 492
PEP 484 introduces type annotations for Python 3. Type annotations can improve code readability for both people and code analysis tools. They make the code better and safer. In this report we will discuss why this is so.
We briefly review what type annotations are and move on to practical examples of where they can be useful and what they give. Next, we discuss some useful tips on writing and using annotations. We will see that in most cases simple type annotations based on classes and built-in collections are sufficient for the public APIs of our libraries. We will also discuss what type annotations can provide for third-party libraries.
Nowadays the world is drowning in textual information. All these years, existing technologies have focused on storing and structuring data. But what to do if we need to make a decision in real time using this data? I’ll be happy to share my experience in developing web applications using full-text search, which are written in Python using the Django framework, and also talk about the results of research from various search engines and integration of web applications with haystack and elasticsearch. Also talk about the future of full-text search in the context of Django 1.9 and Python.
The story goes about how we implemented in Django the same mechanism as Advanced Searching in JIRA, about the internal structure of such solutions (writing DSL, parsing grammars, etc.), why and how we use them on our project.
NumPy is the basic library for scientific computing in Python. On the basis of NumPy arrays, many libraries have been built for various fields of science: machine learning, symbolic computations, and work with biological data. However, in order to use NumPy effectively, you have to slightly change your approach to writing code. We'll see why Python loops work slowly and how the use of vector operations from Numpy improves performance.
One of the most unpleasant things in a programmer’s work is routine. Unfortunately, it cannot be avoided - the work on maintaining the existing code forces us to make the same changes to several files at once. As the project grows, this task grows to dozens of edits in hundreds of files, forcing even the most patient programmer to think about the meaning of life. But what to do? Automate it! In this report, we will talk about different ways to work with source code from Python, and explain why this is fun.
In this report, we will review the memory management subsystem in the most popular Python implementation, CPython. Procedures for allocating memory and garbage collection, ways of interacting with them from the Python code, as well as typical errors that may interfere with their normal operation will be considered. We will also demonstrate a series of simple experiments that prove that working with memory in Python really happens this way and not otherwise.
I'll talk a little about the available methods to lower the IoT entry threshold, talk about the capabilities of the “alljoyn” protocol, one of the most popular IoT protocols, and show with an example of this device how to use the DeviceHive D-Bus Framework from any managed devices that you have at hand, devices that operate on the "alljoyn" protocol.
• The secret composition of celery or an informal introduction to the architecture of Celery;
• We select the ingredients for the sample. How to log and debug queues correctly;
• Take care of a long finish. How to ensure stable operation of queues without downtime and loss of tasks;
• Preparing for the culinary competition. How to avoid racing when performing tasks;
• Several professional chef recipes for setting up celery for large projects.
In most web applications, we somehow interact with the database. It stores information, and we, as programmers, write code that converts this data into html pages or API interfaces. And since the single point of truth is the database, then we need to cache the WEB applications based on its state. In my report, I will talk about the methods of caching, based on the ORM. How you can cache database queries and functions that depend on them. And most importantly, about how such a cache is correctly invalidate.
Pyramid is an extremely flexible framework ideally suited to those developers who know exactly what they need. The other side of its flexibility is the absence of the only correct, approved by the authors, ways to solve problems. In this report, I will share my time-tested recipes for building a RESTful API on Pyramid.
To maintain our standards for both coding style and its semantics, we often use linters. However, adding your own rules to them is quite difficult. Therefore, in the majority of teams, the means in no way or almost do not adapt in any way to either the specific project or the team as a whole. The code review allows you to partially close this flaw, however, sometimes you just want to poke a finger and say "never do that again." I will talk about my research project on how to make the automation of such situations simple and therefore flexible and practical.
In the report we will look at the creation of the setup.py and setup.cfg files. It will be useful to anyone who develops> 1 projects interacting with each other, copies into each new project a folder with favorite functions, installs projects using git clone, copy, rsync and etc., does not write extensions due to the complexity of deployment, writes console scripts using def main () ... if name == 'main' ..., avoids packaging because projects are too dependent on each other (changes in one lead to changes everywhere).
Source: https://habr.com/ru/post/265953/
All Articles