Week 27 / 2023

Mohamed Saif published on
20 min, 3805 words

Categories: fluent-python networking

Tags: python

Networking: Day 2

  • IP Packet

IP Packet

  • TCP/IP Protocol Suite
  • IP (Internet Protocol) and TCP (Transmission Control Protocol) are both protocols within the TCP/IP protocol suite, but they serve different functions and operate at different layers of the networking stack.
  • 1. IP (Internet Protocol):
  • IP is a network layer protocol responsible for addressing and routing packets across networks.
  • It provides the means to encapsulate data into packets, assign unique IP addresses to devices, and determine how packets are routed from source to destination.
  • IP is connectionless and unreliable, meaning it does not establish a dedicated connection before transmitting data and does not guarantee delivery or order of packets. It focuses on best-effort delivery.
  • IP does not provide mechanisms for flow control, error recovery, or congestion control.
  • IPv4 and IPv6 are the two main versions of IP currently in use.
  • 2. TCP (Transmission Control Protocol):
  • TCP operates at the transport layer of the TCP/IP protocol suite.
  • TCP is a connection-oriented protocol designed for reliable and ordered data delivery between applications.
  • It establishes a connection between the sender and receiver before data transfer, using a three-way handshake process.
  • TCP breaks data into segments, assigns sequence numbers to each segment, and ensures that segments are received in the correct order and without errors.
  • It provides mechanisms for flow control, congestion control, and error recovery through acknowledgment and retransmission of lost or corrupted segments.
  • TCP is commonly used for applications that require reliable and ordered data transmission, such as web browsing, email, file transfer, and streaming.
  • In summary, IP is responsible for addressing and routing packets across networks, while TCP is responsible for reliable and ordered data delivery between applications. IP operates at the network layer, focusing on addressing and routing, while TCP operates at the transport layer, providing connection-oriented, reliable, and ordered data transfer.

Day 3

  • Network interface card (NIC):
  • In Linux, a network interface refers to a software component that allows a system to connect to a network. It represents a physical or virtual network connection through which data is sent and received. Network interfaces in Linux are commonly referred to as network cards or NICs (Network Interface Cards).
  • ifconfig: The ifconfig command is used to configure network interfaces and display information about them. This will display information for all active network interfaces, including their names (e.g., eth0, wlan0), IP addresses, netmasks, hardware (MAC) addresses, and other relevant details.
  • The "en0" NIC (Network Interface Controller) is a network interface name commonly used on macOS systems. It represents the first Ethernet interface on the system.
  • Given an IP address and subnet mask
  • To calculate the network address, perform a bitwise "AND" operation between the IP address and the subnet mask. The result will be the network address.
  • To calculate the broadcast address, perform a bitwise "OR" operation between the inverse of the subnet mask and the network address. The result will be the broadcast address.
  • Total Number of Hosts: Subtracting 2 from the total number of possible IP addresses in the subnet will give you the total number of usable hosts. The subtraction accounts for the network address and the broadcast address.
  • Usable IP Range: The usable IP range consists of all the IP addresses between the network address and the broadcast address, excluding those two addresses. It represents the range of IP addresses available for assignment to hosts in the subnet.
  • use the subnet mask to calculate the subnet size or the total number of usable hosts in the network.
  • UDP: User Datagram Protocol
  • a connectionless and unreliable transport protocol within the Internet Protocol (IP) suite. It provides a simple, minimal overhead, and low-latency communication mechanism between networked applications.
  • Unlike TCP (Transmission Control Protocol), UDP does not establish a reliable, ordered, and error-checked connection between the sender and receiver.
  • Connectionless: UDP does not establish a dedicated connection before transmitting data. Each UDP datagram is treated as an independent packet and can be sent without prior setup.
  • Unreliable: UDP does not guarantee delivery, ordering, or error-checking of the transmitted data. It is up to the application layer to handle any necessary reliability mechanisms if required.
  • Low overhead: UDP has minimal overhead compared to TCP. It does not have the additional control mechanisms like sequence numbers, acknowledgments, or congestion control, which makes it lightweight and suitable for applications with low-latency requirements.
  • Datagram-based: UDP transmits data in individual packets called datagrams. Each datagram contains the necessary information, such as source and destination port numbers, to identify the intended application endpoints.
  • Broadcast and multicast support: UDP allows sending data to multiple recipients using broadcast or multicast addresses. This feature is useful for applications like streaming media or real-time communication where data needs to be efficiently distributed to multiple receivers.
  • Suitable for real-time applications: Due to its low overhead and connectionless nature, UDP is commonly used in real-time applications that require fast data transmission, such as VoIP (Voice over IP), video streaming, online gaming, and DNS (Domain Name System) lookups.
  • tcpdump:
  • allows you to capture and analyze network traffic in real-time or save it to a file for later analysis. Tcpdump provides detailed information about individual packets, such as source and destination IP addresses, protocols, port numbers, and packet payload.
  • tcpdump -i wlp3s0: capture packets on a specific network interface.
  • -i: interface
  • Tcpdump provides several filters to capture specific packets. 1. port-based, port <port>. 2. IP-based filters, src <ip> or dst <ip>. 3. protocol-based filters, tcp or udp or icmp. 4. logical operators, and, or, not.
  • nc:
  • (netcat) command is a versatile utility used for network communication and troubleshooting. It provides a wide range of capabilities, such as establishing TCP/UDP connections, listening for incoming connections, port scanning, and data transfer.
  • nc <host> <port>: connect to a specific host and port. Establishing a TCP connection
  • nc -l <port>: listen for incoming connections on a specific port.
  • nc -u <host> <port>: connect to a specific host and port. Establishing a UDP connection

Day 4

  • TCP (Transmission Control Protocol):
  • one of the core protocols of the Internet Protocol Suite (commonly referred to as TCP/IP).
  • responsible for providing reliable, ordered, and error-checked delivery of data packets over an IP network.
  • TCP is a connection-oriented protocol, which means it establishes a virtual connection between the sender and receiver before data transmission.
  • Reliable Delivery: TCP ensures reliable delivery of data by implementing mechanisms such as acknowledgments, sequence numbers, and retransmission of lost or corrupted packets. If a packet is lost or damaged during transmission, TCP detects it and requests the sender to retransmit the missing or corrupted data.
  • Connection-oriented: TCP establishes a logical connection between the sender and receiver before data transmission. The connection involves a three-way handshake process, where the sender and receiver exchange control packets to establish and synchronize the connection state.
  • Ordered Delivery: TCP guarantees that data packets are received in the same order they were sent. This ordering is achieved by assigning sequence numbers to each packet and reassembling them at the receiving end based on these sequence numbers.
  • Flow Control: TCP incorporates flow control mechanisms to prevent overwhelming the receiver with data. The receiver can notify the sender about its available buffer space, allowing the sender to adjust the rate of data transmission accordingly.
  • Congestion Control: TCP includes congestion control mechanisms to prevent network congestion and ensure fair sharing of network resources. It dynamically adjusts the transmission rate based on network conditions, detecting congestion through packet loss and delays.
  • Full-Duplex Communication: TCP supports full-duplex communication, allowing simultaneous bidirectional data transfer between the sender and receiver. Each end can both send and receive data independently.
  • Port-based Communication: TCP uses port numbers to identify specific processes or services running on a device. The combination of IP address and port number forms a unique endpoint for TCP communication.
  • Three-way handshake:
  • The three-way handshake is a key process in establishing a TCP connection between a client and a server. It ensures that both ends agree to establish a reliable connection before data transmission begins.
  • Step 1. SYN (Synchronize):
  • The client initiates the connection by sending a TCP packet with the SYN (Synchronize) flag set.
  • The packet contains a randomly generated initial sequence number (ISN) to identify the client's sequence space.
  • Step 2: SYN-ACK (Synchronize-Acknowledge):
  • Upon receiving the SYN packet, the server responds with a TCP packet that has both the SYN and ACK (Acknowledgment) flags set.
  • The server generates its own random ISN and acknowledges the client's ISN by incrementing it by 1.
  • The server also includes its own initial sequence number to establish its sequence space.
  • Step 3: ACK (Acknowledgment):
  • Finally, the client acknowledges the server's response by sending an ACK packet.
  • The ACK packet has the ACK flag set and acknowledges the server's ISN by incrementing it by 1.
  • At this point, the connection is established, and both client and server are ready to exchange data.
  • Sqeunce Number:
  • TCP uses sequence numbers to identify and reorder data packets. Each packet contains a sequence number that indicates its position in the sequence of packets.
  • Sequence numbers in TCP play a crucial role in ensuring reliable and ordered delivery of data packets between the sender and receiver.
  • purpose:
  • Each TCP segment (packet) contains a sequence number field that uniquely identifies the position of the data within the stream.
  • Sequence numbers are used to order the received data at the receiver's end, allowing the receiver to correctly reassemble the original data stream.
  • They also assist in detecting missing, duplicate, or out-of-order packets during transmission.
  • initial sequence number (ISN)
  • The Initial Sequence Number (ISN) is a randomly generated number chosen by both the sender and receiver during the TCP connection establishment.
  • ISN is used to initiate the sequence numbering for subsequent data packets exchanged between the two endpoints.
  • The choice of a random ISN enhances security by making it difficult for attackers to predict or manipulate the sequence numbers.
  • Sequence Number Increment:
  • For each data segment sent, the sequence number is incremented by the number of bytes in the segment's payload.
  • The receiver uses the sequence number to determine the ordering of incoming segments and detect any missing or out-of-order segments.
  • The acknowledgement (ACK) number in TCP is the next expected sequence number, indicating that all previous bytes have been successfully received.
  • Reliable Data Delivery:
  • By utilizing sequence numbers, TCP ensures reliable delivery of data by retransmitting any lost or unacknowledged packets.
  • If the sender does not receive an acknowledgment (ACK) within a specified timeout period, it retransmits the unacknowledged segment.
  • The receiver uses the sequence numbers to detect any missing or duplicate segments and requests retransmission when necessary.
  • Window Size and Flow Control:
  • TCP's window size parameter is closely related to sequence numbers.
  • The window size indicates the number of bytes the receiver is currently willing to accept.
  • As the receiver consumes data, it advances the ACK number and advertises an updated window size to the sender.
  • The sender uses the window size information to control the flow of data, ensuring it does not overwhelm the receiver's buffer.
  • Congestion Control:
  • a critical feature of TCP that helps prevent network congestion and ensures fair utilization of network resources. It aims to maintain optimal network performance by dynamically adjusting the transmission rate based on the network conditions.
  • Congestion Window (CWND):
  • The congestion window (CWND) represents the number of unacknowledged bytes that a sender can transmit before it expects to receive acknowledgments.
  • Initially, the CWND is small, allowing the sender to cautiously probe the network's capacity. It gradually increases as acknowledgments are received.
  • If the sender detects congestion, it reduces the CWND to reduce the transmission rate and prevent further congestion.
  • Slow Start:
  • During the initial phase of a TCP connection or after a period of inactivity, TCP utilizes a mechanism called slow start to probe the available network capacity.
  • The sender starts with a small CWND and exponentially increases it for every acknowledgment received, effectively probing the network for available bandwidth.
  • Congestion Avoidance:
  • Once the congestion window grows beyond a certain threshold (usually based on the network conditions), TCP transitions from slow start to congestion avoidance.
  • In congestion avoidance, the sender increases the CWND linearly, adding one segment for every round-trip time (RTT) acknowledgment received.
  • Congestion Detection:
  • TCP detects network congestion by monitoring packet loss. If an acknowledgment is not received within a specified timeout period, TCP infers that congestion has occurred.
  • Packet loss can also be indicated by the reception of duplicate acknowledgments (indicating possible segment loss).
  • Congestion Response:
  • Upon detecting congestion, TCP employs various congestion control mechanisms to mitigate the issue and reduce the transmission rate.
  • TCP performs congestion control by reducing the CWND, which in turn reduces the amount of data sent and decreases the network load.
  • TCP uses additive increase and multiplicative decrease to adjust the CWND: it adds a smaller value to the CWND when congestion is not detected, but multiplies it by a fraction (such as 0.5) when congestion is detected.
  • Explicit Congestion Notification (ECN):
  • TCP can also utilize an optional feature called Explicit Congestion Notification (ECN) to enhance congestion control.
  • ECN allows network devices to mark IP packets to indicate congestion, providing early congestion notification to TCP endpoints, which can react accordingly.
  • Network Address Translation (NAT):
  • a technique used in computer networking to enable the translation of IP addresses between different network domains.
  • It allows multiple devices within a private network to share a single public IP address when connecting to the Internet.
  • commonly implemented in routers or firewalls.
  • Private and Public IP Addresses:
  • Private IP addresses are used within a local network and are not routable over the Internet. Examples of private IP address ranges include 192.168.0.0/16, 172.16.0.0/12, and 10.0.0.0/8.
  • Public IP addresses are globally unique and routable on the Internet. They are assigned by Internet Service Providers (ISPs) and are used to identify devices or networks on the Internet.
  • NAT Translation Types:
  • Static NAT: In static NAT, a one-to-one mapping is created between a private IP address and a public IP address. It allows devices with private IP addresses to be accessed from the Internet using their corresponding public IP addresses.
  • Dynamic NAT: In dynamic NAT, a pool of public IP addresses is assigned, and private IP addresses are dynamically mapped to available public IP addresses from the pool. This allows multiple devices to share a limited number of public IP addresses.
  • Port Address Translation (PAT): PAT, also known as NAT Overload, is a type of dynamic NAT that uses port numbers in addition to IP addresses for translation. It maps multiple private IP addresses to a single public IP address by assigning unique port numbers to each connection.
  • Translation Process:
  • When a device from the private network initiates a connection to the Internet, the NAT router replaces the private IP address with its own public IP address (and port number in the case of PAT).
  • The NAT router maintains a translation table that keeps track of the mappings between private IP addresses and their corresponding public IP addresses.
  • When the response packets return from the Internet, the NAT router uses the translation table to replace the public IP address and port number with the appropriate private IP address, allowing the response to reach the correct device within the private network.

Day 5

  • DNS (Domain Name System):
  • a hierarchical and distributed naming system used to translate human-readable domain names into IP addresses that computers understand.
  • It serves as a crucial component of the Internet's infrastructure, enabling users to access websites, send emails, and utilize various network services using memorable domain names instead of numerical IP addresses.
  • In summary, DNS translates domain names into IP addresses, allowing users to access resources on the Internet more conveniently. It functions through a hierarchical and distributed system of DNS servers, from the local resolver to the root servers, TLD authoritative servers, and finally the authoritative servers for specific domains. This hierarchical structure enables efficient and reliable resolution of domain names to IP addresses.
  • 1. DNS Hierarchy:
  • DNS operates in a hierarchical structure, forming a tree-like topology. At the top of the hierarchy are the root servers, which provide information about the top-level domains (TLDs) such as ".com", ".org", ".net", and country-code TLDs like ".uk" or ".fr".
  • Below the TLDs are domain registrars and authoritative DNS servers responsible for specific domains. These authoritative servers hold the authoritative DNS records for their respective domains.
  • 2. DNS Resolution Process:
  • When a user enters a domain name in a web browser, their computer initiates a DNS resolution process to obtain the corresponding IP address.
  • The local DNS resolver (typically provided by the Internet Service Provider) is the first point of contact. If the resolver has the requested information in its cache, it returns the IP address directly. Otherwise, it proceeds with the resolution process.
  • Recursive DNS Resolution:
  • If the local resolver doesn't have the requested information, it contacts other DNS servers in a recursive manner to resolve the query.
  • The local resolver sends the query to the root servers, asking for the authoritative server responsible for the TLD of the requested domain.
  • The root servers respond with a referral to the TLD's authoritative servers.
  • TLD and Authoritative DNS Servers:
  • The local resolver then contacts the TLD's authoritative servers to obtain the IP address of the authoritative server responsible for the next level of the domain.
  • This process continues until the authoritative server for the specific domain is reached.
  • Authoritative DNS Response:
  • The authoritative DNS server receives the query and responds with the requested information, such as the IP address associated with the domain.
  • The response flows back through the hierarchy of DNS servers until it reaches the local resolver and is finally returned to the user's computer.
  • DNS Caching:
  • At each step of the DNS resolution process, DNS servers may cache the obtained information to improve performance and reduce the load on the network.
  • Caching allows subsequent queries for the same domain to be resolved faster, as the information is retrieved from the local cache instead of starting the resolution process from scratch.
  • At the top of the hierarchy are the root servers. They form the foundation of the DNS system and are responsible for storing information about the top-level domains (TLDs), such as ".com", ".org", and country-code TLDs like ".uk" or ".fr".
  • Below the root servers are the authoritative DNS servers for each TLD. They maintain information about the domains registered under their respective TLDs.
  • Authoritative DNS servers for individual domains or subdomains exist further down the hierarchy. They hold the specific DNS records for the corresponding domains, such as A records, MX records, and others.
  • Distributed:
  • The DNS system is distributed across numerous DNS servers worldwide. These servers are geographically distributed and operated by various organizations and entities, including internet service providers (ISPs), domain registrars, and hosting companies.
  • Each level of the hierarchy has its own set of DNS servers responsible for storing and serving the DNS records of the corresponding domain or zone.
  • When a DNS resolver needs to resolve a domain name, it queries the nearest available DNS server in its network, which in turn queries other DNS servers in the hierarchy until the requested information is found.
  • The root servers are strategically distributed across different locations worldwide to ensure redundancy and improve the system's resilience.
  • The 13 root servers, labeled A to M, are operated by various organizations and institutions. These root servers use anycast technology, which allows multiple instances of the same IP address to be distributed across different physical locations. Anycast ensures that DNS queries are automatically routed to the nearest available instance of a root server, minimizing latency and improving performance.
  • Domain Registrars: Domain registrars are organizations that manage the reservation of domain names. They are responsible for registering and maintaining domain names on behalf of their customers. GoDaddy, Namecheap, and Google Domains are examples of domain registrars.
  • Authoritative DNS Servers:
  • Authoritative DNS servers are the final source of truth for the DNS records of a specific domain or subdomain.
  • When a DNS resolver receives a query for a domain name, it contacts the authoritative DNS servers responsible for that domain to obtain the necessary DNS information.
  • Authoritative DNS servers store and serve the DNS records, including A records, MX records, CNAME records, and others, for the corresponding domain.
  • They respond to DNS queries with the accurate and up-to-date DNS information, enabling the resolution of domain names to their associated IP addresses or other relevant data.
  • Authoritative DNS servers are typically managed and operated by domain owners, web hosting companies, or DNS hosting providers.
  • DNS Records:
  • A (Address) Record:
    • The A record maps a domain name to an IPv4 address. It is used to resolve a domain name to its corresponding IP address.
    • Example: example.com. IN A 192.0.2.1
  • AAAA (IPv6 Address) Record:
    • The AAAA record maps a domain name to an IPv6 address. It is used for domains with IPv6 connectivity.
    • Example: example.com. IN AAAA 2001:0db8:85a3:0000:0000:8a2e:0370:7334
  • CNAME (Canonical Name) Record:
    • The CNAME record creates an alias or canonical name for a domain. It redirects queries for the alias to the original or canonical domain name.
    • Example: www.example.com. IN CNAME example.com
  • MX (Mail Exchange) Record:
    • The MX record specifies the mail server responsible for handling email delivery for a domain. It directs incoming emails to the correct mail server.
    • Example: example.com. IN MX 10 mail.example.com
  • TXT (Text) Record:
    • The TXT record holds arbitrary text information associated with a domain. It is often used for various purposes, including domain verification, SPF (Sender Policy Framework) records, and other informational text.
    • Example: example.com. IN TXT "v=spf1 mx -all"
  • NS (Name Server) Record:
    • The NS record indicates the authoritative name servers responsible for a domain. It specifies the DNS servers that hold the authoritative DNS records for the domain.
    • Example: example.com. IN NS ns1.example.com
  • SRV (Service) Record:
    • The SRV record specifies the location of a specific service within a domain. It is commonly used for services like VoIP, instant messaging, and other network services that rely on service discovery.
    • Example: _service._protocol.example.com. IN SRV 10 0 5060 server.example.com
  • To use DNS records, you typically manage them through the control panel or DNS management interface provided by your domain registrar or DNS hosting provider. From there, you can add, edit, or delete the desired records for your domain.
  • It's important to note that DNS record changes may take some time to propagate across the DNS system. This propagation period can vary from a few minutes to several hours, during which DNS servers worldwide update their caches with the new DNS record information.
  • TLS (Transport Layer Security):