Week 43 / 2022

Quotes

• Despite all the code boot camps and programming being taught in elementary schools, the computer itself is often treated as an afterthought — it’s increasingly becoming invisible in the discussions of raising the next generations of computer scientists.
• It’s a lot easier to say something about perfection than to actually achieve it.

command

• to copy from file (use case, public ssh key): xclip -sel clip < ~/.ssh/gitlab.pub

OS

• the i386 instruction set, programming model, and binary encodings are still the common denominator for all 32-bit x86 processors, which is termed the i386-architecture, x86, or IA-32, depending on context.
• x86 is a family of complex instruction set computer (CISC) instruction set architectures initially developed by Intel based on the Intel 8086 microprocessor and its 8088 variant. The 8086 was introduced in 1978 as a fully 16-bit extension of Intel's 8-bit 8080 microprocessor, with memory segmentation as a solution for addressing more memory than can be covered by a plain 16-bit address. The term "x86" came into being because the names of several successors to Intel's 8086 processor end in "86", including the 80186, 80286, 80386 (i386) and 80486 processors.
• x86-64 is the 64-bit extended architecture of x86,
• OpenVMS, often referred to as just VMS,[9] is a multi-user, multiprocessing and virtual memory-based operating system. It is designed to support time-sharing, batch processing, transaction processing and workstation applications.

Problem/ Needs Assessments (graduation project)

• self-taught is overrated because:

1. It requires an enormous amount of self-discipline that many people don't have including knowing how to manage your time, be consistent, and avoid distractions.
2. There are just too many resources & learning paths and it is very challenging to create a learning path without any tech experience.

• great option for those who are self-disciplined and know exactly what they want to learn and ideally have a deadline to accomplish it by.
• consuming redundant content..
• Self-Taught Programming is the easiest path to start but the hardest to finish.
• Learning completely on your own without structure is really tough and can be ineffective. Needing a structure does not mean you need a typical bootcamp/college.
• If one is not careful they can spend their entire life following tutorials.
• Stuff that is useless to my area of practice.
• because I felt like I didn't know enough and that I needed to know more before doing anything.
• experience and practice is what matters the most.
• there are two big reasons people go with self teaching:

1. Cheaper than college/bootcamp
2. Flexibility

• FINISH YOUR PROJECTS.
• Ultimately we're all self-taught, just with varying degrees of structure, accountability, and resources. It's really a question of the quality of material and the motivation to keep engaging with it.
• are missing out on teachers which will provide guidance, correct your errors, they will grade you, they will hold you accountable.
• the gap between what they taught and what they expected was insurmountable.
• These Harvard kids have office hours and mentors to help them and they still struggle with the course, and they're Harvard students.
• Build your own projects constantly to absorb the material and not get stuck in tutorial hell
• app-ideas

EdgeDB

• Projects are intended to make local development easier! They only exist on your local machine and are managed with the CLI. When deploying EdgeDB for production, you will typically pass connection information to the client library using environment variables. The same is true for all EdgeDB client libraries
• You may want to initialize a project that points to a remote EdgeDB instance (docker for examble). you just need to create an alias for the remote instance using edgedb instance link.
• edgedb project unlink: This leaves the instance running but effectively “uninitializes” the project.
• edgedb project unlink -D: to delete the instance

dockerizing fastapi and edgedb

• how to choose the base image? Usually, the slim debian versions are the ones I would prefer in this case, to get a tradeoff between stability, image size and familiarity. article
• The “official” Docker python image is based off Debian 11, so it has access to the same set of up-to-date packages.
• Volumes (anonymous vs named) and bind mount? container data presistance. each DB has a default directory for storing its data (edgedb: var/lib/edgedb/data)

docker volumes

• Every Docker container gets its own non-persistent storage. It’s automatically created, alongside the container, and it’s tied to the lifecycle of the container.
• non-presiestant data, on Linux systems, it exists somewhere under /var/lib/docker/<storage-driver>/ as part of the container.
• At a high-level, you create a volume, then you create a container, and you mount the volume into it.
• Volumes are first-class citizens in Docker. Among other things, this means they are their own object in the API
• Third-party drivers are available as plugins. These can provide advanced storage features, and integrate external storage systems with Docker.
• All volumes created with the local driver get their own directory under /var/lib/docker/volumes on Linux
• you can mount a volume into a new container using docker container run with the --mount flag.
• anonymous: the data will be visible, but not after we remove the container Also, it’s not accessible by other containers. It is helpful if we want to persist data temporarily. created inside /var/lib/docker/volume local host directory. the container will create an anonymous volume by default because it’s specified inside the db Dockerfile itself.
• a dangling volume? docker volume ls -qf dangling=true, to remove them use: docker volume rm $(docker volume ls -qf dangling=true)
• Named: persist data after we restart or remove a container. Also, it’s accessible by other containers. These volumes are created inside /var/lib/docker/volume local host directory. edgedb-data:/var/lib/edgedb/data, the first field is a unique name of the volume on a host machine. The second part is the path in the container.
• named volumes and bind mounts are the same, except the named volumes can be found under a specific host directory, and bind mounts can be any host directory.

commands

• to update pyenv: cd ~/.pyenv then git pull.
• error when pyenv install -v 3.11.0, ERROR: The Python ssl extension was not compiled. Missing the OpenSSL lib?, the solution was:

1. brew uninstall --force openssl
2. brew cleanup -s openssl
3. brew cleanup --prune-prefix
4. brew install openssl
5. echo 'export PATH="/home/linuxbrew/.linuxbrew/opt/openssl@3/bin:$PATH"' >> ~/.bashrc
6. CFLAGS="-I$(brew --prefix openssl)/include" LDFLAGS="-L$(brew --prefix openssl)/lib" pyenv install -v 3.11.0, where brew --prefix openssl = /home/linuxbrew/.linuxbrew/opt/openssl@3

CH3: AsyncIO Walk-Through

PyCon 2017: Miguel Grinberg - Asynchronous Python for the Complete Beginner

• Asynchrony, in computer programming, refers to the occurrence of events independently of the main program flow and ways to deal with such events.
• A practical definition of Async: A style of concurrent programming in which tasks voluntarily release the CPU during waiting periods, so that other tasks can use it.
• No OS intervention.
• One process, one thread
• HOW? the ability to “suspend” and “resume”, and schedular (the event-loop)

Intro

• Asyncio provides another tool for concurrent programming in Python (threads, and multiprocessing are the other tools/paradigms)
• The asyncio API in Python is complex because it aims to solve different problems for different groups of people (end-user developers and framework developers)

Quickstart: the building blocks

• You only need to know about seven functions to use Asyncio, can be summarized as follows:

1. Starting the asyncio event loop
2. Calling async/await functions
3. Creating a task to be run on the loop
4. Waiting for multiple tasks to complete
5. Closing the loop after all concurrent tasks have completed

• Most of your Asyncio-based code will use the run()
• Anywhere you call get_event_loop() will give you the same loop instance each time, as long as you’re using only a single thread (99% of the time you’re going to use only a single, main thread for your app). If you’re inside an async def function, call asyncio.get_running_loop() instead
• loop.create_task(coro) schedules your coroutine to be run on the loop. The returned task object can be used to monitor the status of the task.
• if you use asyncio.run(), none of these steps are necessary, they are all done for you.
• run() creates a new event loop every time you call it?
• asyncio in Python exposes a great deal of the underlying machinery around the event loop—and requires you to be aware of aspects like lifecycle management.
• how to run blocking functions(which would have blocked the main thread and prevented your event loop from running.). The thing about cooperative multitasking is that you need all I/O-bound functions to…well, cooperate, and that means allowing a context switch back to the loop using the keyword await.
• Until there is more widespread support for async def functions, you’re going to find that using such blocking libraries is unavoidable. For this, asyncio provides the executer?
• returns a Future?
• loop.run_in_executor(None, blocking_func) Sometimes you need to run things in a separate thread or even a separate process: this method is used for exactly that?

The Tower of Asyncio

• Tier 1: Coroutines
• Tier 2: The Event Loop (asyncio provides both a loop specification, AbstractEventLoop, and an implementation, BaseEventLoop.)
• the uvloop project provides a much faster loop implementation than the one in the asyncio standard library module.
• Tier 3, and Tier 4: futures and tasks, which are very closely related; they’re separated only because Task is a subclass of Future, but they could easily be considered to be in the same tier
• Tier 5: represents the facilities for launching, and awaiting on work that must be run in a separate thread, or even in a separate process.

Coroutines

• coroutine object and asynchronous function?
• it looks like a regular function, except that it begins with the keywords async def.
• The inspect module in the standard library can provide much better introspective capabilities than the type() built-in function.
• ..and it is only when this function is evaluated.. (generators and routines)

def g()
    yield 100

type(g) # function
gen = g()
type(gen) # generator

###
async def c():
    return 100

type(c) # function
coro = c()
type(coro) # coroutine

• A coroutine is an object that encapsulates the ability to resume an underlying function that has been suspended before completion.
• How Python is able to “switch” execution between coroutines.
• A coroutine is initiated by “sending” it a None coro.send(None). Internally, this is what the event loop is going to be doing to your precious coroutines
• When a coroutine returns, what really happens is that a StopIteration exception is raised. (The StopIteration exception is the normal way that coroutines exit)
• the event loop will be responsible for driving coroutines with these low-level internals. From your point of view, you will simply schedule coroutines for execution on the loop, and they will get executed top-down, almost like normal functions.
• await always takes a parameter and will accept only a thing called an awaitable (Coroutine or Any object implementing the await() special method)
• when you call task.cancel(), the event loop will internally use coro.throw() to raise asyncio.CancelledError inside your coroutine.
• CancelledError is raised inside the task-wrapped coroutine when tasks are cancelled (being injected into the coroutine from outside, by the event loop)
• task cancellation is nothing more than regular exception raising (and handling)
• what happens if after cancellation, we just go right back to awaiting another awaitable?

Event Loop

• The event loop in asyncio handles all of the switching between coroutines, as well as catching those StopIteration exceptions—and much more, such as listening to sockets and file descriptors for events.
• recommended asyncio.get_running_loop(), and discouraged asyncio.get_event_loop().
• You do not need to pass an explicit loop parameter through all your functions.
• Most of us will only ever need (and want!) a single loop instance running in a single thread. This is nearly the entire point of async programming in the first place.
• get_running_loop() (the recommended method) will always do what you expect: because it can be called only within the context of a coroutine, a task, or a function called from one of those, it always provides the current running event loop, which is almost always what you want.

Python Internals resources

• PG, CPython internals: A ten-hour codewalk through the Python
• Python behind the scenes: a list of resources