Week 36 / 2022

PSQL

• \c [database]
• \dt list relations/tables
• the input device is not a TTY when cat Chinook_PostgreSql.sql | docker exec -it postgresql psql -U saif -w -d chinook?? Solution is to remove the -it?
• docker-compose exec postgres psql -U postgres
• SELECT * FROM [TABLE] returns nothing? => JUST PUT THIS SHIT ;

Essential SQLAlchemy

• INSERT: ins = [table].insert().values(...); print(str(ins)) # print sql statements
• one() vs first(): one() Return exactly one result or raise an exception. first() Return the first result of this Query or None if the result doesn’t contain any row.
• DataAccessLayer?

web Scrapping

• Each site usually has a robots.txt on the root of their domain. This is where the website owner explicitly states what bots are allowed to do on their site.
• using standard urllib: html_str = urlopen(<url-str>).read().decode(); urlopen() returns a HttpResponse object. read returns a sequence of bytes. decode returns a string format of this sequence. (2) ways to work with the html_str: (first: str methods). (second: regexes) .findall(<pattern>, <str_to_test>) returns a list of the matched strings. Often, you use .search() to search for a particular pattern inside a string. This function is somewhat more complicated than.findall() because it returns an object called a MatchObject that stores different groups of data. This is because there might be matches inside other matches, and .search() returns every possible result. .search() either returns None or a MatchObject. sub(), which is short for substitute, allows you to replace text in a string that matches a regular expression with new text.
• tutor: learndatasci
• HTML parser: beautifulsoup4. (1) create a beautifulsoup4 object to start working with the html. .get_text() extract all the text from the document and automatically remove any HTML tags. find_all(<tag: img>) to return a list of all instances of that particular tag (list of tag objects).
• "wb" stands for "write bytes". This let's us avoid any encoding issues when saving.
• When we pass our HTML to the BeautifulSoup constructor we get an object in return that we can then navigate like the original tree structure of the DOM.: we can find elements using names of tags, classes, IDs, and through relationships to other elements, like getting the children and siblings of elements.
• (1: find elements and data). select_one: returns a single element. select: returns a list of elements. Both of these methods use CSS selectors to find elements (1) get a tag use the naked name for the tag. (2) .class-name. (3) #ID. (4) .class-name child(a/img/tr..etc) get childs of the class-name parent: E.g. to get <div class="temp"><a></a></div> use select_one('.temp a'). Note the space between .temp and a.

Selenium

• selenium.webdriver module provides all the WebDriver implementations.
• from selenium.webdriver.common.keys import Keys provide keys in the keyboard like RETURN, F1, ALT etc.
• from selenium.webdriver.common.by import By used to locate elements within a document.
• element = driver.find_element(By.ID/NAME/XPATH/CSS_SELECTOR, "<value>")
• (Problem 1): 'WebDriver' object has no attribute 'find_element_by_id', Use find_element(BY.[], "").

C: chapter 1

• <stddef.h> to access 'NULL'

• include guard?

• Normally you are at liberty to give functions whatever names you like, but ``main'' is special - your program begins executing at the beginning of main. This means that every program must have a main somewhere.

• A sequence of characters in double quotes, like "hello, world\n", is called a character string or string constant.

• \t for tab, \b for backspace, \" for the double quote and \\ for the backslash itself.

• ntrast with float, which means floating point, i.e., numbers that may have a fractional part. The range of both int and float depends on the machine you are using

• in C, as in many other languages, integer division truncates: any fractional part is discarded.

• Each % construction in the first argument of printf is paired with the corresponding second argument, third argument, etc.; they must match up properly by number and type, or you will get wrong answers.

• right-justified? __ augment each %d in the printf statement with a width.

• We were unable to use 5/9 in the previous version because integer division would truncate it to zero. However, so 5.0/9.0 is not truncated because it is the ratio of two floating-point values.

• If an arithmetic operator has integer operands, an integer operation is performed. If an arithmetic operator has one floating-point operand and one integer operand, however, the integer will be converted to floating point before the operation is done. Nevertheless, writing floating-point constants with explicit decimal points even when they have integral values emphasizes their floating-point nature for human readers.

• Width and precision

• Format Specifier: %o for octal, %x for hexadecimal, %c for character, %s for character string and %% for itself.

• for loop (initialization, condition, increment)

• The initialization, condition and increment can be any expressions.

• The for is usually appropriate for loops in which the initialization and increment are single statements and logically related, since it is more compact than while and it keeps the loop control statements together in one place.

• One way to deal with magic numbers is to give them meaningful names. A #define line defines a symbolic name or symbolic constant to be a particular string of characters: #define name replacement list. Notice that there is no semicolon at the end of a #define line.

• The model of input and output supported by the standard library is very simple. Text input or output, regardless of where it originates or where it goes to, is dealt with as streams of characters. A text stream is a sequence of characters divided into lines; each line consists of zero or more characters followed by a newline character.

• Each time it is called, getchar reads the next input character from a text stream (or buffer) and returns that as its value.

• We must declare c to be a type big enough to hold any value that getchar returns. EOF is an integer defined in <stdio.h>, but the specific numeric value doesn't matter as long as it is not the same as any char value.

• The parentheses around the assignment (c = getchar()) != EOF, within the condition are necessary. The precedence of != is higher than that of =, which means that in the absence of parentheses the relational test != would be done before the assignment =. Without the parentheses c = getchar() != EOF => c = (getchar() != EOF)

• If the body of the for loop is empty, because all the work is done in the test and increment parts. And since the grammatical rules of C require that a for statement have a body. The isolated semicolon, called a null statement, is there to satisfy that requirement. We put it on a separate line to make it visible.

• %f for both float and double; %.0f suppresses the printing of the decimal point and the fraction part, which is zero.

• getchar() is a Buffered function?

• Difference between "" and '' in C? __ when c == "\n", where c is an int, you got this warning: warning: comparison between pointer and integer. But when c == '\n' it's fine? single quotes identify a single character, while double quotes create a string literal (double quotes are null-terminated strings (char *)). 's' is a single a character literal ('s' an integer, representing the numerical value of the letter 's' in the machine’s character set), while "s" is a string literal containing an 'a' and a null terminator (that is a 2 char array), in other words, an array of characters, two characters long, consisting of 's' followed by '\0'. Note that in C, the type of a character literal is int, that is sizeof 'a'.

• Again, A character written between single quotes represents an integer value equal to the numerical value of the character in the machine's character set. This is called a character constant, although it is just another way to write a small integer. So, for example, 'A' is a character constant; in the ASCII character set its value is 65, the internal representation of the character A. Of course, 'A' is to be preferred over 65: its meaning is obvious, and it is independent of a particular character set.

• a = b = c = 0: an assignment is an expression with the value and assignments associated from right to left. It's as if we had written a = (b = (c = 0))

• Expressions connected by && or || are evaluated left to right, and it is guaranteed that evaluation will stop as soon as the truth or falsehood is known.

• so that if c is a digit, c-'0' is the numeric value of that digit? __ in asciitable The decimal representation for the character '0' -> '9' is 48 -> 57. Most, if not all compilers, follow this conversion table. By subtracting '0' (most likely the number 48), you essentially turn the character representation of the variable c into a number representation.

• By definition, chars are just small integers, so char variables and constants are identical to ints in arithmetic expressions. This is natural and convenient; for example c-'0' is an integer expression with a value between 0 and 9 corresponding to the character '0' to '9' stored in c variable.

• We will generally use parameter for a variable named in the parenthesized list in a function. The terms formal argument and actual argument are sometimes used for the same distinction.

• Since main is a function like any other, it may return a value to its caller, which is in effect the environment in which the program was executed. Typically, a return value of zero implies normal termination; non-zero values signal unusual or erroneous termination conditions.

• It is an error if the definition of a function or any uses of it do not agree with its prototype. parameter names need not agree. Indeed, parameter names are optional in a function prototype, so for the prototype we could have written

• In C, all function arguments are passed by value.'' This means that the called function is given the values of its arguments in temporary variables rather than the originals. This leads to some different properties than are seen with call by reference'' languages like Fortran or with var parameters in Pascal, in which the called routine has access to the original argument, not a local copy.

• When necessary, it is possible to arrange for a function to modify a variable in a calling routine. The caller must provide the address of the variable to be set (technically a pointer to the variable), and the called function must declare the parameter to be a pointer and access the variable indirectly through it.

• The story is different for arrays. When the name of an array is used as an argument, the value passed to the function is the location or address of the beginning of the array - there is no copying of array elements. By subscripting this value, the function can access and alter any argument of the array.

• The purpose of supplying the size of an array in a declaration is to set aside storage.

• getline puts the character '\0' (the null character, whose value is zero) at the end of the array it is creating, to mark the end of the string of characters. This conversion is also used by the C language. Terminated with a '\0' to mark the end.

• Copy character array: copy 'from' into 'to':

    i = 0;
    while ((to[i] = from[i]) != '\0')
      ++i;
  

• Each local variable in a function comes into existence only when the function is called, and disappears when the function is exited. This is why such variables are usually known as automatic variables, following terminology in other languages. We will use the term automatic henceforth to refer to these local variables. (Chapter 4 discusses the static storage class, in which local variables do retain their values between calls.)

• An external variable must be defined, exactly once, outside of any function; this sets aside storage for it. The variable must also be declared in each function that wants to access it; this states the type of the variable. The declaration may be an explicit extern statement or may be implicit from context

• Before a function can use an external variable, the name of the variable must be made known to the function; the declaration is the same as before except for the added keyword extern.

• In certain circumstances, the extern declaration can be omitted. If the definition of the external variable occurs in the source file before its use in a particular function, then there is no need for an extern declaration in the function.

• If the program is in several source files, and a variable is defined in file1 and used in file2 and file3, then extern declarations are needed in file2 and file3 to connect the occurrences of the variable. The usual practice is to collect extern declarations of variables and functions in a separate file, historically called a header, that is included by #include at the front of each source file. The suffix .h is conventional for header names. The functions of the standard library, for example, are declared in headers like <stdio.h>.

• sizeof(int) -> long unsigned int = 4 Bytes = 32 bits

• sizeof(float) -> long unsigned int = 8 Bytes = 64 bits

C: chapter 2

• The type of an object determines the set of values it can have and what operations can be performed on it.

• Don't begin variable names with underscore, however, since library routines often use such names.

• At least the first 31 characters of an internal name are significant.

• For external names, the standard guarantees uniqueness only for 6 characters and a single case.

• There are only a few basic data types in C (char, int: typically reflecting the natural size of integers on the host machine, float, double)

• qualifiers: (int: short, long, signed, unsigned); The intent is that short and long should provide different lengths of integers where practical. (char: signed, unsigned)

• The standard headers <limits.h> and <float.h> contain symbolic constants for all of these sizes, along with other properties of the machine and compiler.

• The standard library is not part of the C language proper, but an environment that supports standard C will provide the function declarations and type and macro definitions of this library.

• what is <header-file.h>? a file contains C (function declarations?) and (macro definitions?) to be shared between several source files.. Including a header file is equal to copying the content of the header file. A simple practice in C or C++ programs is that we keep all the constants, macros, system wide global variables, and function prototypes in the header files and include that header file wherever it is required.

• All preprocessing directives begin with a # symbol.

• A macro is a fragment of code that is given a name. You can define a macro in C using the #define preprocessor directive. function-like macros:

    // Every time the program encounters circle_area(argument), it is replaced by (3.14 * argument * argument).
    #define circle_area(r) (3.14 * r * r)
  

• #include <system_header_file.h>: This variant is used for system header files. It searches for a file named system_header_file.h in a standard list of system directories. You can prepend directories to this list with the -I option.

• #include "your_header_file.h": This variant is used for header files of your own program. It searches for a file named your_header_file.h first in the directory containing the current file, then in the quote directories and then the same directories used for <your_header_file.h>. You can prepend directories to the list of quote directories with the -iquote option.

• Included files are not limited to declarations and macro definitions; those are merely the typical uses. Any fragment of a C program can be included from another file

• Search Path: By default, the preprocessor looks for header files included by the quote form of the directive #include "file" first relative to the directory of the current file, and then in a preconfigured list of standard system directories. For the angle-bracket form #include <file>, the preprocessor’s default behavior is to look only in the standard system directories.

• The most commonly-used option is -I[dir], which causes dir to be searched after the current directory (for the quote form of the directive) and ahead of the standard system directories.

• The directory list printed by the -v option reflects the actual search path used by the preprocessor?

• Once-Only Headers: a header file should be read only once.

• #include_next?

• The header files declaring interfaces to the operating system and runtime libraries often cannot be written in strictly conforming C. Therefore, GCC gives code found in system headers special treatment?

• A macro is a fragment of code which has been given a name. Whenever the name is used, it is replaced by the contents of the macro. There are two kinds of macros. They differ mostly in what they look like when they are used. Object-like macros (it looks like a data object in code that uses it) resemble data objects when used, function-like macros resemble function calls.

• When the preprocessor expands a macro name, the macro’s expansion replaces the macro invocation, then the expansion is examined for more macros to expand. For example,

    #define A B
    #define B 100
    // A -> B -> 100
  

• A function-like macro is only expanded if its name appears with a pair of parentheses after it. If you write just the name, it is left alone

• stringizing: When a macro parameter is used with a leading ‘#’, the preprocessor replaces it with the literal text of the actual argument, converted to a string constant.

• Look!

    #define xstr(s) str(s)
    #define str(s) #s
    #define foo 4
    /*
      str (foo) → "foo"
      xstr (foo) → xstr (4) → str (4) → "4"
    */
  

• Unsigned constants are written with a terminal u or U, and the suffix ul or UL indicates unsigned long.

• Floating-point constants contain a decimal point (123.4) or an exponent (1e-2) or both; their type is double, unless suffixed. The suffixes f or F indicate a float constant; l or L indicate a long double.

• A character constant is an integer, written as one character within single quotes, such as 'x'.

• Character constants participate in numeric operations just as any other integers, although they are most often used in comparisons with other characters

• The character constant '\0' represents the character with value zero, the null character. '\0' is often written instead of 0 to emphasize the character nature of some expression, but the numeric value is just 0.

• A string constant, or string literal, is a sequence of zero or more characters surrounded by double quotes

• String constants can be concatenated at compile time: "Hello, " "World!" ->>> "Hello, World!"

• Technically, a string constant is an array of characters. Terminated with the null character '\o'

• Enumuration: a list of interger consecutive (unless otherwise) values: enum days_num{sat, sun, mon, tus, wes, tur, fri} -> (0,1,2,3,4,5,6)

• The const declaration can also be used with array arguments, to indicate that the function does not change that array: int strlen(const char[]); The result is implementation-defined if an attempt is made to change a const.

• Expressions connected by && or || are evaluated left to right, and evaluation stops as soon as the truth or falsehood of the result is known.

• the precedence of != is higher than assignment

• A common use of ! is in constructions like if (!valid) (read nicely (``if not valid'')) rather than if (valid == 0)

• When an operator has operands of different types, they are converted to a common type according to a small number of rules.

• A char is just a small integer, so chars may be freely used in arithmetic expressions. This permits considerable flexibility in certain kinds of character transformations.

• The standard header <ctype.h>, defines a family of functions that provide tests and conversions that are independent of character set.

• For portability, specify signed or unsigned if non- character data is to be stored in char variables.

• the lower'' type is promoted to the higher'' type before the operation proceeds.

• Floating point numbers in C use IEEE 754 encoding.

• Conversion rules are more complicated when unsigned operands are involved. The problem is that comparisons between signed and unsigned values are machine-dependent, because they depend on the sizes of the various integer types.

• Finally, explicit type conversions can be forced (coerced) in any expression, with a unary operator called a cast. In the construction (type name) expression

• Note that the cast produces the value of n in the proper type; n itself is not altered.

• declaration causes automatic coercion of any arguments when the function is called.

• in-line assignment causes segmentaton fault int a = b = c = 0;???? __ because (b and c) are not declared

• In both cases, the effect is to increment n. But the expression ++n increments n before its value is used, while n++ increments n after its value has been used.

• This means that in a context where the value is being used, not just the effect, ++n and n++ are different. If n is 5, then x = n++; sets x to 5, but x = ++n; sets x to 6. In both cases, n becomes 6.

• Bitwise operators: may only be applied to integral operands, that is, char, short, int, and long, whether signed or unsigned.

C: chapter 6: Structures

• A structure is a collection of one or more variables, possibly of different types, grouped together under a single name for convenient handling.

• An optional name called a structure tag may follow the word struct. The tag names this kind of structure, and can be used subsequently as a shorthand for the part of the declaration in braces.

• The variables named in a structure are called members.

  struct type_name {
    member_type1 member_name1;
    member_type2 member_name2;
    member_type3 member_name3;
  
  } object_names;
  

• For example:

    struct product {
      int weight;
      double price;
    } ;
  
    product apple;
    product banana, melon;
  

  • This declares a structure type, called product, and defines it having two members: weight and price, each of a different fundamental type

  • This declaration creates a new type (product), which is then used to declare three objects (variables) of this type: apple, banana, and melon. Note how once product is declared, it is used just like any other type.

  • the optional field object_names can be used to directly declare objects of the structure type.

  • In this case, where objectnames are specified, _the type name (product) becomes optional: struct requires either a type_name or at least one name in object_names, but not necessarily both.

  • clearly differentiate between what is the structure type name (product), and what is an object of this type (apple, banana, and melon). Many objects (such as apple, banana, and melon) can be declared from a single structure type (product).

    struct product {
      int weight;
      double price;
    } apple, banana, melon;
  

• A member of a particular structure is referred to in an expression by a construction of the form structure-name.member

• Structures can be nested. One representation of a rectangle is a pair of points.

• There are at least three possible approaches: pass components separately, pass an entire structure, or pass a pointer to it. Each has its good points and bad points.

• If a large structure is to be passed to a function, it is generally more efficient to pass a pointer than to copy the whole structure.

• struct point *pp; says that pp is a pointer to a structure of type struct point. If pp points to a point structure, *pp is the structure, and (*pp).x and (*pp).y are the members.

• The parentheses are necessary in (*pp).x because the precedence of the structure member operator . is higher than *. The expression *pp.x means *(pp.x), which is illegal here because x is not a pointer.

• If p is a pointer to a structure, then p->member-of-structure refers to the particular member. So we could write instead printf("origin is (%d,%d)\n", pp->x, pp->y); Both . and -> associate from left to right

• C provides a compile-time unary operator called sizeof that can be used to compute the size of any object. The expressions sizeof object and sizeof (type name) yield an integer equal to the size of the specified object or type in bytes. (Strictly, sizeof produces an unsigned integer value whose type, size_t, is defined in the header <stddef.h>) An object can be a variable or array or structure. A type name can be the name of a basic type like int or double, or a derived type like a structure or a pointer.

C: chapter 7

• A text stream consists of a sequence of lines; each line ends with a newline character. If the system doesn't operate that way, the library does whatever necessary to make it appear as if it does.
• The simplest input mechanism is to read one character at a time from the standard input, normally the keyboard, with getchar
• In many environments, a file may be substituted for the keyboard by using the < convention for input redirection: if a program prog uses getchar, then the command line prog <infile causes prog to read characters from infile instead.
• otherprog | prog runs the two programs otherprog and prog, and pipes the standard output of otherprog into the standard input for prog.
• Again, output can usually be directed to a file with >filename: if prog uses putchar, prog >outfile will write the standard output to outfile instead. If pipes are supported, prog | anotherprog puts the standard output of prog into the standard input of anotherprog.
• When the name is bracketed by < and > a search is made for the header in a standard set of places (for example, on UNIX systems, typically in the directory /usr/include).

GDB

• Type r or run to run the program.
• GDB has over 1500+ commands!
• To start in neato and highly-recommended GUI mode, start the debugger with gdb -tui.
• Command line arguments! What if you have to get something into argv in your program? Pass them as arguments to the run command when you start execution
• Breakpoints? __ Just starting the debugger to run the program straight through isn't very useful—we need to stop execution and get into stepping mode.
• First, before you issue the run command, you need to set a breakpoint someplace you'd like to stop. You use the break or b command, and specify a location, which can be a function name, a line number, or a source file and line number.
• to list all breakpoints: info breakpoints or the shorter i b
• To clear a breakpoint, use the clear command with the breakpoint location. You can also clear a breakpoint by number with the delete command.
• Additionally, you can enable or disable breakpoints, though these two commands take a breakpoint number as an argument, not a location!
• (That weird stuff at the end about __libc_start_main() shows you that there was another function that called your main() function! It wasn't compiled with debugging information so we can't see the source, but we can still step through it—which we do—and the program exits normally.)
• Now, notice that next steps over function calls. This doesn't mean that function doesn't get called; it means that next will execute the function until it's done, and then return you to the next line in your current function.
• What if you have a function you want to step into from your current function, and trace through that function line-by-line? Use the step (or s) command to do this. Let's say you're tired of single stepping, and just want the program to run again. Use the continue (or c) command to continue execution.
• What if the program is running but you forgot to set breakpoints? You can hit CTRL-C and that'll stop the program wherever it happens to be and return you to a "(gdb)" prompt.
• If you have some variables you wish to inspect over the course of the run, you can display them, but only if the variable is currently in scope.

Date and Times in C

• The header <time.h> declares types and functions for manipulating date and time.
• local vs calender vs GMT?
• good article
• time_t vs tm?
• Broken-down time is a human-readable version of the calendar time. The struct tm data type is used for broken-down time. The localtime() functions converts a simple calendar time into a broken-down time. It takes the current timezone into account.

Typer

• If there's one subcommand in the app @app.command(): the --help not showing it as a subcommand, it woulf be the default commad of the typer app?

Agile: OKR

• I will ________ as measured by ____________.
• An OKR is a popular management strategy that defines objectives and tracks results. It helps create alignment and engagement around measurable goals.
• OKRs have two important parts: The objective you want to achieve and the key results, which are the way you measure achieving the objective.

القضاء والقدر