TIL

TIL

By. Me

1. NAND Flash Interface

Interface between cpu and chips. Why? I was wondering about best filesystem for my external SSD, because obviously write-cached external device is bad. I always just unplug my drive without unmounting and ejecting, so I’m always afraid about data corruption. So, i was searching about flash-friendly file systems, and I found JIFFS2 or UBIFS?. JIFFS was familliar, I have seen it from many embedded board projects. It was raw NAND chip friendly. We need FAL? (Flash Abstract Layer) when we use flash chips. But we dont care about these in ssds. Why? Because SSD Controllers embedded in ssd itself manages this abstraction and also controls wearing, also caching in local DRAM. So, we dont need to care about these things when we use SSD. But in embedded board that normally doesnt have seperate controller rather than just attaching NAND chips directly to CPU. Like NOR chip I used several times, NAND is also connected directly. I used QSPI when connecting NOR chip, but SPI was not sufficient for high-speed high data-rate NAND chips. We need parallel communicaion. Even though I have seen a lot of NAND projects, I never saw this part of firmware controlling and parsing NAND signals. I’ve seen NAND controller firmware that does block-level jobs, but never this interfacing things. So I searched a bit about NAND interface, and I found that there arent many standards in this part. There was ONFI (Open NAND Flash Interface) that defines standard interface for NAND chips, but not many major chips supported, maybe. I saw some documentations like Micron’s, and well, that was complex. I dont know yet about hardware timing and things like that, not about that low-level of things. Well but I was interested about that things. Also I learned about Eth PHY and interfaces like MII, GMII… It defines signal level interface and again I am not familiar with these things. And, this was all thing I searched about.

2. PHY?

What is PHY? Physical Layer from OSI Layers. Usually indicates the chip/part where parses incoming signal (via RF or cable) and then send it to next layer in standardized way. ADC. Analog singal to digital data. Raw Analog <–> Amplifing, Filtering, Equalizing <–> Decoding, Encoding, Error checking (CRC), Clock Data Recovery.. <–> Data Link

PHY Designs? Line Driver / Amplifier for better analog resolution PLL (Phase-Locked Loops) for CDR and matching clocks from incoming clock/data signal. DSP (Digital Singal Processor) for processing raw data to known format data.

Eth PHY. MAC. (Media Access Control) <-> PHY via MII GMII (Media Independant Interface) PHY just transform Ethernet Cable signal into data packet. From MAC, actual content of data is interpreted, MAC distinguishes each devices, de/encapsulates.

Line Driver? Part of transmission line signal. Make sure right signal arrived well. Matching impedence.. etc… (how? and what more?)

DSP Design? For example in Eth PHY what is processed in DSP? Deserialization? ( Parsing differential signals? => line driver maybe )

3. D-Bus?

D-Bus is IPC system for Linux, it provides structualized way for apps to communicate with each other.

Desktop Bus System.

System-bus and session-bus

System services <-> root application / system users.

User session <-> user applications and users.

=> Event-driven communication / Object-oriented

Messaging. - calls / returns / error / signal

Object - interface -> method -> property -> signal

Method = (call, return, error, signal)

D-Bus router = message broker / router (just like other message forward/proxy protocols)

Method Calls -> request of service? RPC?

Signals. kernel signal in userspace. send signal to application.

Objects .. Message = Serialized bytearray of any signal / method calls. It is like RPC. Multi-multi RPC-bus

D-Bus Message structure.

Byteorder / Message Type / Flags / Protocol Version / Length / UUID / Header / Body.

How and where can I use D-Bus in my application?

Communication with low-level D-Bus daemon(server).

Ex. MediaPlayer - Implement Play/Pause/Stop/Volume for user to control via DE’s common interface i.e. play button on taskbar? Extensions? Anyway structulized way of control.

Ex. DE can get information about songs now playing, via D-Bus.

Ex. System monitoring application e.g. runCat.

Ex. Caffeine-ng -> prevents machine from sleeping.

Ex. Solaar -> Controls mouse/keyboard in low-level. Needs D-Bus.

How?

Define D-Bus interface via XML, signals. methods, props. (export to bus?) -> Implement by library -> Connect to D-Bus (sync with daemon?) -> Do something.

Session vs System D-Bus

Session ? e.g. Push notification to DE. we need to send it via d-bus.

System ? e.g. Power Event, kernel-level type of event managing.

4. ABI?

For me it wasnt clear enough about definition of ABI.

Application Binary Interface. Defines how data structures from machine code. System library / binary compatability. e.g. Data Type, Size, Layout, Alignment, ..

  1. In Linux, struct dirent{} and corresponding of other OS differ. It is data structure mismatch, which is part of ABI. Also, typing like int, long may be 32bits, 64bits,,, though it usually doesnt differ in modern system.
  2. Alignment of memory.
  3. Calling convention … How argument is passed. Via “eax, ebx, ecx…” or in stack? and also return values.
  1. Binary object format, ELF, PE, Mach-O is also part of ABI.
  2. glibc, musl, … this libc mismatch is also part of ABI incompatibility. Function signatures, syscall wrappers, symbol versioning, …

API is just defining high-level approach of data exchange, ABI is more lower level approach of binary applications talk to each other via binary data structures and object conventions.

API read(fd, mode, bytes) can be same but where parameter is passed (in ASM level) may differ.

5. Linux Kernel’s Driver Management

< Android HAL >

Android. HAL (Hardware Abstraction Layer) Why? Linux kernel already includes a lot of driverset, why additional userspace device driver manager is needed?

Android can be configured by a lot different parts, compared to PC or server. Like lte modem, wifi chip, screen chip, etc. Linux’s mainline kernel cant manage all this many chips.

Its like alternative universal driver package. Variety of hardware config, but allow modular and consistent interface. (In Linux world, there are also Abstract Layers like HAL, like ALSA/PA or libinput or even FUSE is Abstract layer if you wanna say so…)

Its bridge between hardware driver and Android Framefork layer, and Android Framework driver provides universal interface. Android Apps can use those peripherals through standardized Android API (like Camera API or sensors, GPS..). Hardware vendors makes API driver? (can we say that’s server?) from their hardware to Android Manger class.

Linux Userspace Driver

“Userspace Driver.”

How exactly userspace drivers can access and use physical peripherals? Peripherals usually use “cpu port” in x86, and “memory map” in ARM. Unlike kernel that actually can access these without any restrictions, they somehow need to access to raw port or raw memory.

UIO (Userspace I/O)

MMIO (Memory-Mapped I/O)

or, map hardware register’s address into specific userspace virtual address.

Drivers and CPU Peripherals

CPU has its own preconfigured ports (or pins), e.g. PCI, USB, and I/O buses like SPI or I2C. Signal through this pins are handled by CPU as the part of SoC. If we take a look at CPU’s datasheet, we can see how pins are assigned for specific signals. CPU pins are assinged to RAM-related pins, JTAG, GPIO, PCIe, I2C, UART, SPI, and so on. Beside complex I/O like USB, PCIe, SATA that requires high datarate and strict signal timing, less complex I/O like UART or I2C can use GPIO pins for communication. GPIOs are versatile and can be programmed to communicate with various interfaces. This way of using software instead of dedicated hardware is called “bit banging”, and we can easily see this on like Arduino.

But there arent many interfaces that doesnt requires strict timing, only UART or basic I2C can be bit-banged. In most time, dedicated controller hardware is required. These hardware blocks can handle the timing and electric signals or even validation of data, thus ensures higher throughput and reliability.

CPU has limited amount of pins, and we need to connect tremendous amount of devices. External hardware controllers and buses are required to expand I/O. These controllers can often interface with many devices on the same bus (like multiple SPI or I2C devices) and serialize the communications.

Serialization is the key of efficient communication. CPU technology is eveolved, and we can use very high speed buses. With these high-speed buses serial communication has advantage over paralell communication, which was primarily used when we only had low-speed buses.

Well, there was northbridge and southbridge long CPU ago, CPU needed additional chipset for all high speed interface, like RAM or PCIe. Then northbridge is directly connected to CPU via “Front side bus”. {{ }} .

Currently, just those bridges are integrated into CPU as SoC] or just one chipsets. Those chipsets connects to CPU as PCIe.

During the boot process, the OS will initialize MMU, but in early stage of booting, the CPU may operate in a mode where the MMU in not yey enabled or is configured to provide a direct mapping, so that kernel can to permissive tasks. Once OS is loaded, is first sets up MMU with kernel’s own memory mapping setup, and also necessary permissions among address spaces.

In case of MMIO, OS will configure MMU to create direct mapping between requesting process’s VA space and device’s physical address that its register mapped into. Also disabling CPU caching is done for this address range.

So, How does OS configure MMU?

Linux’s driver problem

6. Container & Sandboxing

Flatpak → OSTree Commits , combining..

GNOME

app / org.gnome.Sdk ← org.gnome.Platform/master ← org.freedesktop.Platform/23.08 ,, Minimal linux environment.

bwrap, sandbox environment.

No userspace process isolation so I can see /proc in host, with /proc/{pid}/root which listing it actaully shows me root inside sandbox. via mountinfo I can see it is bindmount from /newroot → /root.

/newroot is on tmpfs, meaning that there arent any actual ‘files’ mapped in host, so I cant find it.

/newroot is setted up by bwrap, no “base” image.

All flatpak commits, including Platform, is bind mounted to container’s folders. e.g. org.gnome.Platform’s files/ will be ro-mounted on /usr@container, and flatpak invokes bwrap with

symlink /bin /usr/bin, symlink /lib /usr/lib, mkdir /dev, mkdir /tmp, mkdir /sys, mkdir /app … without need of mounting image on root, but it sets up like minimal linux on embedded firmware (e.g. busybox linux)

App will be ro-binded in /app, platform will be ro-binded on /usr, user-data will be saved at XDGs, $HOME.

What is bootc? It is exactly same as ostree-native-containers, its original of that.

fakeroot

LD_PRELOAD, geteuid() → return 0. It fools program that uid is 0, but anyways user cant access to real root’s permissions. Just fooling programs that “only allows running as root internally”.

containered environment,

rootless → current user becomes root, any additional users inside container gets virtual uid, in host its just ghost id. real root becomes nobody and of course its not accessible by user.

In here we need “rooless” unpacking. Container Image fill have files with root ownership in their tar archive. So we need fakeroot to unpack image to filesystem, so that “root” files in tar to be extracted.

7. libc and compat layer

8. ggml?

9. eBPF?

10. Serialization & Reflection

11.