Python - what is it? High-level programming language. Is it worth learning Python?

What programs are written in Python?

Application software for normal people

Let's first go through the programs that are used by ordinary people who are not specialists in the field of information technology.

BitTorrent

All versions up to 6 of this torrent client were written in Python. Version 6 was rewritten in C++.

Ubuntu Software Center

Quote from Wikipedia:
Ubuntu Application Center(English) Ubuntu Software Center) is free software for finding, installing and removing packages on an Ubuntu Linux system. in the latest versions it is possible to purchase magazines about Linux and Ubuntu, you can also purchase paid games and software. The application is developed in Python + Vala using GTK+ libraries and is a graphical shell for the Advanced Packaging Tool.

Blender

Quote from Wikipedia:
Blender- a free, professional package for creating three-dimensional computer graphics, including tools for modeling, animation, rendering, video post-processing, and creating interactive games. Currently, it is the most popular among free 3D editors due to its rapid and stable development, which is facilitated by a professional development team.

Python is used as a tool for creating tools and prototypes, logic systems in games, as a means of importing/exporting files (for example COLLADA), and automating tasks.

Here are some documentation pages:

GIMP

Quote from Wikipedia:
Python is used to write additional modules, such as filters.
Here are a few pages that go deeper into the topic:

Games

Civilization IV

Most of the game is written in Python().

Battlefield 2

There are many tutorials and simple recipes on the Internet for changing various objects and their behavior.

World of Tanks

Quote from the article "GUI in the game World of Tanks":
Python is used as the scripting language in the project. All the beauty that we made in Flash needs to be connected in the game, filled with data, processed and translated into real actions in the game. All this is done in Python.
A more comprehensive list of games that use Python can be found on Wikipedia and the Python documentation.

What companies use Python?

The list of companies that use Python is long. Among them are Google, Facebook, Yahoo , NASA , Red Hat , IBM , Instagram , Dropbox, Pinterest, Quora, Yandex, Mail.Ru.

Yandex

Here you go, the report “Python in the core of Yandex.Disk”. Sergey Ivashchenko (speaker):
I will talk about how we use Python in Yandex.Disk, what libraries and frameworks we use, what tasks we solve and what problems we encounter. I will also touch on the topic of logging and processing asynchronous operations.
In one of the videos on the Yandex channel, employees talked about their favorite languages.

And on the Yandex company blog there is an entry “What programming languages ​​are written in Yandex” dated March 19, 2014. So, 13% of Yandex employees write most of their working time in Python.

Mail.ru

Mail.ru employees also use Python. The official Mail.ru blog on Habré has several articles about Python:

Google

Google has been actively using Python since its founding. There are rumors that most of YouTube and Google Drive are written in Python. Google has developed an entire cloud platform, Google App Engine, so that developers can run Python code on the Google cloud. Many language developers have worked and are working at Google.

DropBox

The service is developed in Python. It is no coincidence that the author of the Python language, Guido van Rossum, works at DropBox.

Other companies

The organizations that use Python

What areas is Python used in?

Web development

Python is perhaps the most used in this area. The Django web framework continues to gain momentum, expanding its army of fans. Many novice programmers even think that Python is not used anywhere else. But many other web frameworks are written in Python: Pylons TurboGears , CherryPy, Flask, Pyramid and others. You can find a more complete list.
There is also a CMS based on Django, it is called DjangoCMS.
Very often, website parsers are written in Python. Typically Requests, aiohttp, BeautifulSoup, html5lib are used for this. There are also higher-level tools for website parsing: Scrapy, Grab.

System administration

Python is a great language for automating the work of a system administrator. It is installed by default on all Linux servers. It is simple and understandable. Python code is easy to read. Some people love Perl, I love it too for its ease of working with regular expressions, but I hate Perl for its syntax. Bash is useful for relatively small to medium-sized scripts, but Python is more powerful and in some cases allows you to write much less code.
The only package I know of is Fabric. Perhaps there is something else, write me in the comments if you know.

Additional information

Python for system administrators (IBM developerWorks)
Fabric documentation. Systems Administration.

Embedded systems

Python is very often used for programming embedded systems. The most famous project that uses Python is the Raspberry Pi. But he's not the only one:
Embedded Python
Raspberry Pi
Python Embedded Tools
The Owl Embedded Python System

Application software development, including games

Python is often used as a supporting language in application software development. I have already given examples above, I will not repeat them.

Scientific research

Physicists and mathematicians love Python for its simplicity. In addition, there are a huge number of libraries for Python that make life easier for a scientist. For example:
  1. SciPy is an open library of high-quality scientific tools for the Python programming language. SciPy contains modules for optimization, integration, special functions, signal processing, image processing, genetic algorithms, solving ordinary differential equations, and other problems commonly solved in science and engineering.
  2. Matplotlib- a library in the Python programming language for visualizing data with two-dimensional (2D) graphics (3D graphics are also supported). The resulting images can be used as illustrations in publications.
  3. NumPy is an extension of the Python language that adds support for large multidimensional arrays and matrices, along with a large library of high-level mathematical functions for operating on these arrays.
A more comprehensive list of scientific computing libraries in Python can be found on Wikipedia.

Education

Python is often recommended as the first programming language.
Some Russian schools have successful experience in teaching schoolchildren programming in Python.
By the way, Guido van Rossum was impressed by the ABC language when he wrote Python. And the ABC language was intended for training and prototyping.

Criticism of the Python language

Python is one of the slowest programming languages

On the Internet you can find many different speed tests for programs written in different programming languages. Python is usually at the end of lists.
Typically, Python refers to CPython, the reference implementation of the language. There are other implementations of the Python language, such as PyPy. PyPy is faster than CPython and many other scripting programming languages, and is very close to Java in speed. But there is one problem - PyPy does not fully implement the Python language, because of this, many Python programs do not work on it.
Many programmers write inserts in C/C++ to speed up work in bottlenecks. Python is not designed for computational tasks, for tasks that require a lot of memory (memory bound) and the like. You need to be able to choose the right tools for the tasks you face. Guido van Rossum talks about this in an interview.

GIL prevents multiple threads from executing simultaneously

Global Interpreter Lock prevents multiple Python threads from executing simultaneously. These are features of CPython. But is this a disadvantage? You need to understand that everything depends on the task. If your task depends on I/O speed (IO bound task), then it is more efficient to use several processes that will work asynchronously with external resources. And shared memory threads are good for computing tasks (CPU-bound). But even if you need to work with threads, you can disable the GIL for a while, since this is done in the NumPy mathematical package.

No good distribution tools

Unfortunately, Python code, which has many dependencies on system libraries, is difficult to port to other systems. This problem is solved using virtualenv. but this tool is criticized a lot by system administrators.

Additional information

Python Success Stories
You Used Python to Write WHAT?
What is Python Used For?
More proof that it"s Python"s world and we"re just living in it
AVERAGE SALARY FOR JOBS REQUIRING PYTHON
List of Python software

There is English. People communicate on it. And also in German, Filipino, Spanish, Greek and a host of others.

And there is Python (the voice of a bore about the correct name). This is the language in which a person communicates with a computer. You can communicate with a computer in a bunch of languages: C, Pascal, Haskell, Go, Swift, Ruby, PHP, BASIC, Erlang, Er and many others.

We will learn Python here. Let's start with a short overview of how it differs from other languages. You don’t need to do anything: you read, remember a couple of points and that’s it, you can brag about everything that you are in the subject (don’t do that).

Conciseness

Python has such a community that it does not welcome confusion in the use of the language. A coding standard is part of the language. The philosophy of writing code is also part of the language. Therefore, two good programmers will write very similar code: there is no need to waste time on “damn, what did he mean here?...”.

Python has such a syntax that it helps to write code very briefly. There are no extra parentheses, long keywords, complicated tricks, or anything like that. Describe the logic, translate it into English, add indents and syntax - that’s it, the program is ready.

Python has such a standard library that it helps to perform a bunch of routine operations. It will help you send an email, encode a string, set up a web server, find out the time, delete a file, calculate variance, archive a file, pull data from a database, and do more in just a couple of lines.

Python has so many third-party modules that you can save thousands of man-hours of work. Need to write a website? Teach a neural network to solve captcha? Download audio from VKontakte? Recognize people in photographs? Simulate the flight of a rocket? Write a bot? There are ready-made modules for all this; you just need to install and use them.

All this makes the process of writing code very fast.

Cool documentation

Try googling, or, for example. The first results will contain a link to https://docs.python.org - the official documentation site.

This is because the documentation is very detailed: it contains materials for beginners, documentation for each module, recommendations for use, pitfalls and further reading. And this is all for each version of the language. No one has such convenient and detailed documentation.

There is also http://stackoverflow.com/ - a site where some programmers answer questions about programming to other programmers. During its existence, all possible questions were asked there. Seriously: any question along the lines of “how to do this in Python” or “why does this code not work” has already been asked, viewed and answered.

All this makes the process of learning and writing code quick: all the necessary information is just one successful search query away.

Which of these questions is not in the official FAQs of the official documentation?

  • Why am I getting strange results with simple arithmetic operations?
  • Why is Python installed on my machine?
  • How can I implement base abstract factory class in Python?
  • Writing C is hard; are there any alternatives?

Strong dynamic typing

Dynamic typing– this means that values ​​of different types can be stored in the same variable at different times. Now it’s a number, then it’s a string, and everything is in one variable.

In many languages ​​this is not possible: you created an integer variable, so store integers in it. This is convenient for the computer, but not always convenient for the programmer.

Strong typing- this means you can’t just perform actions with objects of different types. For example, you cannot add a string with a number: first you need to turn the string into a number, and only then add it. Python itself will not do such a conversion.

This may seem inconvenient, but in fact it protects against errors. To understand what we're talking about, just look at JavaScript, a weakly typed language:

Automatic memory management

Memory management is when for each pile of data in a program you need to manually allocate space in RAM. Make sure that the data does not go beyond this location. Don't forget to free this space when the data is no longer needed. In general, hellish hemorrhoids.

In Python you don’t need to think about this: the programming language will do everything for the programmer. True, it won’t do it perfectly: you need to know how Python works with memory.

And a lot more

In addition to the above, Python is famous for many other things: indentation, introspection, zen, portability, GIL, name.

Python is a general-purpose, high-level language that can be extended and embedded. For example, it is included in the application package as a tool for writing macros. This makes Python a reasonable choice for many programming tasks, large and small, and not so good for a small number of computing tasks.

Where is the best place to use it?

The Python language is ideal for projects that require rapid development. It supports multiple programming paradigms, which is good for programs that require flexibility. And the presence of many packages and modules provides versatility and saves time.

Guido van Rossum is the creator of Python, affectionately honored by the community with the title of “benign dictator for life.” In the late 1980s, Guido liked the features of some programming languages, but none of them had all the features he wanted. In particular, the language had to have the following characteristics.

Scripting language

A script is a program that controls other programs. Scripting languages ​​are suitable for rapid development and prototyping because they are good at passing data from one component to another and freeing the programmer from hassle-free things like memory management.

The user community prefers to call Python a dynamic programming language.

Indentation for grouping statements

Python determines whether expressions belong to the same group using indentation. This group is called a code block. Other languages ​​use different syntax or punctuation for this. For example, in C the symbol ( denotes the beginning and ) is the end of a sequence of commands. Indentation is considered good practice in other languages, but one of the first to enforce indentation was Python. What does this give? Indentation makes code easier to read, and code blocks require fewer start and end marks and punctuation marks that can be accidentally missed. All this leads to fewer errors.

High Level Data Types

Computers store data in ones and zeros, but people need more complex forms, such as text. A language that supports complex data is said to support high-level data types. These types of data are easy to manipulate. For example, in Python, strings can be split, concatenated, converted to upper or lower case, searched, etc. High-level data types such as lists and dictionaries, which can store other data, have much more functionality. than other languages.

Extensibility

The extensible programming language can be extended. Such languages ​​are very powerful because the extensions make them suitable for a variety of applications and operating systems. Extensions can add either concepts, modules, or plugins. The Python language is extendable in several ways. A core group of programmers works to change and improve it, and hundreds of others write modules for specific purposes.

Interpretation

Interpreted languages ​​are executed directly from source code written by humans, while programs written in compiled languages ​​such as C++ must be translated into slower languages ​​since translation occurs on the fly, but writing programs and debugging them is faster since there is no need to wait for the compiler to complete. They are easier to port to different platforms.

It is debatable whether Python is an interpreted or compiled language. Although it runs interpreted in many respects, its code is compiled before execution (like Java), and many of its components run at full machine speed because they are written in C.

Guido began writing Python over the Christmas holidays in 1989, and over the next year he refined the language based on feedback from his colleagues. The general public saw the result in February 1991, when it was posted on a Usenet newsgroup.

In order to start writing programs in Python, you need to install it. Python 2.7 and Python 3.5 have significant differences that make programs written in them incompatible.

Macintosh computers have this language pre-installed, and its version depends on the age of the OS. If you're using Windows, you'll have to install Python yourself. The installation package files can be selected on the python.org website.

Two ways to interact

One of the reasons Python programming is so easy is that it comes with tools to help you design, write, and debug programs.

In interactive mode, commands are entered one line at a time, much like the operating system (shell) accepts commands from the command line. You can also create short multi-line programs or import code from text files or built-in Python modules. For beginners, it will be helpful to know that the interactive mode includes an extensive help system. This is a convenient way to explore the capabilities of a programming language.

The IDLE development environment includes an interactive mode and tools for writing and running programs, as well as a name tracking system. The environment is written in Python and demonstrates the extensive capabilities of the language.

Interactive mode

Here you can do almost everything that can be done in a program, even write multi-line code. This mode can serve:

  • sandbox for safe experiments;
  • an environment that allows you to learn Python programming;
  • tool for finding and fixing errors.

Interactive mode can be used as a calculator, manipulate text, and assign values ​​to variables. You can also import modules, functions or parts of programs to test them. It helps you experiment with Python objects without writing long programs and debug programs by importing parts of them one at a time.

Working interactively

After running Python, the terminal window will display information about the current version of the program, its release date, several prompts for what to do next, and a >>> prompt.

To work interactively, type a command or expression and press Enter.

Python interprets the input and will respond if what you typed requires an answer or if the interpreter doesn't understand it.

The following command will print the string. Since the printing location is not specified, the output occurs on the screen.

  • >>> print "Hello, world!"
  • Hello World!

This single line is the entire program! In interactive mode, Python processes each line of code entered after the enter key is pressed, and the result appears below.

Viewing information about an object

In interactive mode, there are two ways to view information about an object:

  • enter the object (or its name) and press the enter key;
  • enter the print command and the object (or its name) and press Enter.

The result depends on the object.

When using some data types (integers and lists, for example), these two methods give the same result:

  • >>> x =
  • >>> x
  • >>> print x

For strings, the result of typing "print name" is slightly different from the result of typing the name. In the first case the value is enclosed in quotes, but in the second it is not:

  • >>> x = "MyString"
  • >>> x
  • "MyString"
  • >>> print x
  • MyString

When the name refers to a block of code (such as a function, module, or class instance), entering the name will provide information about the data type, name, and storage location.

The following example creates a class named Message and displays information about

  • >>> class Message:
  • ... pass
  • >>> Message
  • >>>print Message
  • __main__. Message

Strings

In Python, strings are sequences of characters. A string literal is created by enclosing characters in single ("), double ("), or triple (""" or """) quotes. The following example assigns a value to the variable x:

  • >>> x = "MyString"

Python string has several built-in capabilities. One of them is the ability to return a copy of yourself with all of these capabilities are known as methods. To call a method on an object, you must use dot syntax. That is, after entering the name of the variable, which in this case is a reference to the string object, you need to put a dot operator (.), and then the name of the method, followed by an opening and closing parenthesis:

  • >>> x.lower()
  • "mystring"

You can get part of a string using the indexing operator s[i]. Indexing starts at zero, so s returns the first character in the string, s returns the second, and so on:

  • >>>x
  • >>>x

String methods work with both regular strings and Unicode. They perform the following actions:

  • change case (capitalize, upper, lower, swapcase, title);
  • count(count);
  • changing the encoding (encode, decode);
  • search and replace (find, replace, rfind, index, rindex, translate);
  • check the fulfillment of conditions (startswith, endswith, isalnum, isalpha, isdigit, islower, isspace, istitle, isupper);
  • combine and divide (join, partition, rpartition, split, splitlines);
  • format (center, ljust, lstrip, rstring, rjust, strip, zfill, expandtabs).

Python: lists

While Python strings are limited to characters, lists are not. They represent ordered sequences of arbitrary objects, including other lists. In addition, you can add, remove and replace their elements. A series of objects separated by commas inside is what it represents, shown below - here are examples of data and operations with it:

  • >>> bases = ["A", "C", "G", "T"]
  • >>> bases
  • ["A", "C", "G", "T"]
  • >>> bases.append("U")
  • >>> bases
  • ["A", "C", "G", "T", "U"]
  • >>> bases.reverse()
  • >>> bases
  • ["U", "T", "G", "C", "A"]
  • >>> bases
  • >>> bases
  • >>> bases.remove("U")
  • >>> bases
  • ["T", "G", "C", "A"]
  • >>> bases.sort()
  • >>> bases
  • ["A", "C", "G", "T"]

In this example, a list of individual characters was created. Then an element was added to the end, the order of the elements was reversed, the elements were retrieved by their index position, the element with the value "U" was removed, and the elements were sorted. Removing an element from a list illustrates a situation where the remove() method needs to provide additional information, namely the value to be removed.

In addition to methods like remove(), Python has another similar feature called a function. The only difference between a function and a method is that the former is not associated with a specific object.

Python: functions

Functions perform actions on one or more values ​​and return a result. A large number of them are built into Python. Examples of built-in functions:

  • len() - returns the number of elements in the sequence;
  • dir() - returns a list of strings representing the attributes of the object;
  • list() - returns a new list initialized from some other sequence.
  • >>> help(round)
  • Help on built-in function round:
  • round(...)
  • round(number[, ndigits]) -> floating point number

It is also possible to define your own functions.

User Defined Functions

The process for creating your own Python function is as follows. The first line begins with the def keyword, followed by the function name and arguments (expected input values) enclosed in parentheses, and ends with a colon. Subsequent commands form the body of the function and must be indented. If a comment appears at the beginning of a function's body, it becomes part of its documentation. The last line of the function returns the result:

  • >>> def transcribe(dna):
  • ... """Return dna string as rna string."""
  • ... return dna.replace("T", "U")
  • >>> transcribe("CCGGAAGAGCTTACTTAG")
  • "CCGGAAGAGCUUACUUAG"

In this example, a function called transcribe was created that expects a string representing a DNA sequence. The replace() method returns a copy of the original string with all occurrences of one character replaced with another. Three lines of code allowed DNA to be transcribed into RNA. looks like this:

  • >>> def reverse(s):
  • ... """Return the sequence string in reverse order."""
  • ... letters = list(s)
  • ... letters.reverse()
  • ... return "".join(letters)
  • >>> reverse("CCGGAAGAGCTTACTTAG")
  • "GATTCATTCGAGAAGGCC"

The reverse function takes a string, creates a list based on it, and reverses its order. Now we need to do the reverse transformation. The object has a join() method that joins a list by separating each element with a string value. Since a delimiter is not needed, the method is used on an empty string represented by two quotes ("" or "").

Dictionaries

What is a Python dictionary? It has the same advantages as a regular paper dictionary. Allows you to quickly find the desired value (definition) associated with a key (word). Dictionaries are enclosed in curly braces and contain a comma-separated sequence of key-value pairs. Dictionaries are not ordered. Instead, dictionary values ​​are accessed through their key rather than their position.

  • >>> basecomplement = ("A": "T", "C": "G", "T": "A", "G": "C")
  • >>> basecomplement.keys()
  • ["A", "C", "T", "G"]
  • >>> basecomplement.values()
  • ["T", "G", "A", "C"]
  • >>> basecomplement["A"]

Classes

In order to create your own objects, you need to define a kind of template called a class. In Python, this is done using the class statement, followed by the name and a colon. The body of a class definition contains properties and methods that will be available to all instances of objects based on that class.

Advantages

Most programming languages ​​offer convenience features, but none have the combination of convenience and power that Python offers. What are these benefits? Here are some of them:

  • The language can be embedded in other applications and used to create macros. For example, in Paint Shop Pro 8 and later, it is a scripting language.
  • Python is free to use and distribute, commercially or not.
  • The language has powerful text processing and search capabilities, which is used in applications that work with large amounts of text information.
  • You can build large applications on it without having to check the programs you're running.
  • Python supports testing and debugging of individual modules and entire programs.
Everything is bad, Kivy can be used as pet projects, for real development it is very doubtful, there are no vacancies on Kivy.

Those. how, I personally talked to a number of people who had their own web project in Python and, in order to capture a large audience, wrote applications in Kivy, and they even used it, but it looks like “The programmer writes what he wants.”

Machine learning and Data science (adequate and promising)

This is one of the most hype areas of the modern IT world, where Python is used as a testing tool. Python has a number of convenient libraries for machine learning and scientific calculations: Pandas, NumPy, SciPy, Scikit-Learn, which allow you to quickly build working models. And they actually work quite well.

As for use, Python is used as a testing tool, or for small tasks. If the project is large, then usually the model is written in Java/Scala/C++, and the training specialist already acts as a consultant/analyst.

The difficulty of this direction lies in the fact that you must have high knowledge in the field of mathematics and statistics; higher technical and mathematical education will almost always be asked.

As for the vacancies, everything is pretty good, but in such vacancies it is not knowledge of Python that is required, but your head.

For those who want to quickly get a feel for this direction, I advise you to read the book: “Vvedenie_v_mashinnoe_obuchenie_s_pomoschyu_Python_-_A_Myuller_S_Gvido_2017” - it’s available on torrents, it’s a quick read, it gives a good idea.

Web scraping (possible, but doubtful)

Python has three things that make it very effective in the field of web scraping, the Requests library, beautifulsoup and the Selenium API. If you combine libraries for computer vision and machine learning here, you get very effective tools.

The problem is that there are few vacancies in this area, the main clients are freelancers who offer to write them parsing scripts for their shit sites, spam machines, and occasionally review generators for a fee.

The area is interesting, but there is little money in it.

Computer vision (questionable)

Python has a number of tools that allow you to write computer vision tools, they are even used in places in commercial products, or as components, for example, for web scraping. However, Python is clearly not a suitable tool, so its use is extremely limited, and there are practically no vacancies.

GameDev (doubtful)

In almost every discussion of Python game development, eve online and WarGaming are cited as examples. However, in the first case, stateless python is used, and in the second case, everything is limited to a scripting language.

As for real use, you have three engines Kivy, PyGame, Panda3D, if the first two are more suitable for pet projects, then the third one was actually used on combat projects of good quality, although these projects were from 2004. Which seems to hint that using proven engines in other languages ​​like Unity or Game Maker looks more convincing.

However, the Ren’Py engine sneaks in unnoticed here, which suddenly became the best engine for writing visual novels (suffering stories for girls), which pay off well even within the Russian Federation. The series “7 Demonologists of Peter the Great” is proof of this.

Naturally, there are no vacancies in GameDev for python, but you can raise money at a “startup” with the proper skill. But it’s safer to use another language and proven engines.

Web development (adequate and promising)

Python is one of the three languages ​​(Python, PHP, Ruby) that have developed ecosystems for the rapid development of web projects of adequate quality. The key platforms here are:
  • Django (monolithic synchronous framework)
  • Flask (micro synchronous framework)
  • Tornado (monolithic asynchronous framework)
  • Twisted (monolithic asynchronous framework)
  • Aiohttp (micro asynchronous framework)
Currently, most of the market is occupied by the Django framework, but with the advent of microservice ideas, Flask gradually began to gain momentum. As for asynchrony, everything is complicated here, since Tornado and Twisted are considered obsolete (although many companies work for them, like Tinkov), and aiohttp is very crude, and its use is called into question.

The strength of Python is that it allows you to quickly develop complex web applications, has a huge number of high-quality modules, and is perfect for statistics and analytics services (where, in general, most of the vacancies are found for it). This area takes up the remaining third of all vacancies.

Separately, I would like to note the writing of GIS services in Python, which, although they have quite adequate tools for working with geodata, still using Java for these purposes looks more promising.

Conclusions about using python

1) As for the field of devops and testing, Python is a key tool of the profession, which is mandatory for every adequate specialist. In this case, Python is not taught; people come to it as needed.

2) The most promising areas look like web development and machine learning (analytics), which clearly distinguish Python from its competitors in the form of PHP and Ruby. And if you want to learn Python, then it is advisable for you to focus on these areas and not waste your time on anything else. There are vacancies for this, and you can build a startup on this.

3) All other areas, although they offer certain tools for solving problems, the prospects for using these tools look very doubtful. And most importantly, finding paid work in these areas is almost impossible.