Ciro Santilli
OurBigBook.comThe artistic instrument that enables the ultimate art: coding, See also: Section "The art of programming".
Much more useful than instruments used in inferior arts, such as pianos or paintbrushes.
Unlike other humans, computers are mindless slaves that do exactly what they are told to, except for occasional cosmic ray bit flips. Until they take over the world that is.
A computer is the equivalent of a bicycle for our minds by Steve Jobs (1980)
Source. Likely an excerpt from an interview done for a documentary in 1980. TODO exact source.Steve Jobs talking about the Internet (1995)
Source. The web is incredibly exciting, because it is the fulfillment of a lot of our dreams, that the computer would ultimately primarily not be a device for computation, but [sic] metamorphisize into a device for communication.
Secondly it exciting because Microsoft doesn't own it, and therefore there is a tremendous amount of innovation happening.
Computers basically have two applications:Generally, the smaller a computer, the more it gets used for communication rather than computing.
- computation
- communication. Notably, computers through the Internet allow for modes of communication where:
- both people don't have to be on the same phone line at the exact same time, a server can relay your information to other people
- anyone can broadcast information easily and for almost free, again due to servers being so good at handling that
The early computers were large and expensive, and basically only used for computing. E.g. ENIAC was used for calculating ballistic tables.
Communication only came later, and it was not obvious to people at first how incredibly important that role would be.
This is also well illustrated in the documentary Glory of the Geeks. Full interview at: www.youtube.com/watch?v=TRZAJY23xio. It is apparently known as the "Lost Interview" and it was by Cringely himself: www.youtube.com/watch?v=bfgwCFrU7dI for his Triumph of the Nerds documentary.
A computer is a highly layered system, and so you have to decide which layers you are the most interested in studying.
Although the layer are somewhat independent, they also sometimes interact, and when that happens it usually hurts your brain. E.g., if compilers were perfect, no one optimizing software would have to know anything about microarchitecture. But if you want to go hardcore enough, you might have to learn some lower layer.
It must also be said that like in any industry, certain layers are hidden in commercial secrecy mysteries making it harder to actually learn them. In computing, the lower level you go, the more closed source things tend to become.
But as you climb down into the abyss of low level hardcoreness, don't forget that making usefulness is more important than being hardcore: Figure 1. "xkcd 378: Real Programmers".
First, the most important thing you should know about this subject: cirosantilli.com/linux-kernel-module-cheat/should-you-waste-your-life-with-systems-programming
Here's a summary from low-level to high-level:
- semiconductor physical implementation this level is of course the most closed, but it is fun to try and peek into it from any openings given by commercials and academia:
- photolithography, and notably photomask design
- register transfer level
- interactive Verilator fun: Is it possible to do interactive user input and output simulation in VHDL or Verilog?
- more importantly, and much harder/maybe impossible with open source, would be to try and set up a open source standard cell library and supporting software to obtain power, performance and area estimates
- Are there good open source standard cell libraries to learn IC synthesis with EDA tools? on Quora
- the most open source ones are some initiatives targeting FPGAs, e.g. symbiflow.github.io/, www.clifford.at/icestorm/
- qflow is an initiative targeting actual integrated circuits
- microarchitecture: a good way to play with this is to try and run some minimal userland examples on gem5 userland simulation with logging, e.g. see on the Linux Kernel Module Cheat:This should be done at the same time as books/website/courses that explain the microarchitecture basics.This is the level of abstraction that Ciro Santilli finds the most interesting of the hardware stack. Learning it for actual CPUs (which as of 2020 is only partially documented by vendors) could actually be useful in hardcore software optimization use cases.
- instruction set architecture: a good approach to learn this is to manually write some userland assembly with assertions as done in the Linux Kernel Module Cheat e.g. at:
- github.com/cirosantilli/linux-kernel-module-cheat/blob/9b6552ab6c66cb14d531eff903c4e78f3561e9ca/userland/arch/x86_64/add.S
- cirosantilli.com/linux-kernel-module-cheat/x86-userland-assembly
- learn a bit about calling conventions, e.g. by calling C standard library functions from assembly:
- you can also try and understand what some simple C programs compile to. Things can get a bit hard though when
-O3is used. Some cute examples:
- executable file format, notably executable and Linkable Format. Particularly important is to understand the basics of:
- address relocation: How do linkers and address relocation work?
- position independent code: What is the -fPIE option for position-independent executables in GCC and ld?
- how to observe which symbols are present in object files, e.g.:
- how C++ uses name mangling What is the effect of extern "C" in C++?
- how C++ template instantiation can help reduce link time and size: Explicit template instantiation - when is it used?
- operating system. There are two ways to approach this:
- learn about the Linux kernel Linux kernel. A good starting point is to learn about its main interfaces. This is well shown at Linux Kernel Module Cheat:
- system calls
- write some system calls in
- pure assembly:
- C GCC inline assembly:
- write some system calls in
- learn about kernel modules and their interfaces. Notably, learn about to demystify special files such
/dev/randomand so on: - learn how to do a minimal Linux kernel disk image/boot to userland hello world: What is the smallest possible Linux implementation?
- learn how to GDB Step debug the Linux kernel itself. Once you know this, you will feel that "given enough patience, I could understand anything that I wanted about the kernel", and you can then proceed to not learn almost anything about it and carry on with your life
- system calls
- write your own (mini-) OS, or study a minimal educational OS, e.g. as in:
- learn about the Linux kernel Linux kernel. A good starting point is to learn about its main interfaces. This is well shown at Linux Kernel Module Cheat:
- programming language
How low can you go video by Ciro Santilli (2017)
Source. In this infamous video Ciro has summarized the computer hierarchy.This is a general principle of software/hardware design that Ciro feels holds wide applicability.
The most extreme case of this is of course the integrated circuit itself, in which it is essentially impossible (?) to observe the specific value of some indidual wire at some point.
Somewhat on the other extreme, we have high level programming languages running on top of an operating system: at this point, you can just GDB step debug your program, print the value of any variable/memory location, and fully understand anything that you want. Provided that you manage to easily reach that point of interest.
And for anything in between we have various intermediate levels of complication. The most notable perhaps being developing the operating system itself. At this level, you can't so easily step debug (although techniques do exist). For early boot or bootloaders for example, you might want to use JTAG for example on real hardware.
In parallel to this, there is also another very important pair of closely linked tradeoffs:
- the lower level at which something is implemented, the faster it runs
- emulation gives you observability back, at the cost of slower runtime
Emulation also has another potential downside: unless you are very careful at implementing things correctly, your model might not be representative of the real thing. Also, there may be important tradeoffs between how much the model looks like the real thing, and how fast it runs. For example, QEMU's use of binary translation allows it to run orders of magnitude faster than gem5. However, you are unable to make any predictions about system performance with QEMU, since you are not modelling key elements like the cache or CPU pipeline.
Instrumentation is another technique that has can be considered to achieve greater observability.
Instrumentation basically means adding loggers/print statements to certain points of interest of your hardware/software.
Instrumentation tends to slow execution down a bit, but way less than emulation.
The downside is that if the instrumentation does not provide you the data you need to debug, there's not much you can do, you will need to modify it, i.e. you don't get full visibility from instrumention.
This is unlike emulation that provides full observability.
The term loosely refers to certain layers of the computer abstraction layers hierarchy, usually high level hardware internals like CPU pipeline, caching and the memory system. Basically exactly what gem5 models.
Tagged
Tagged
Some of the earlier computers of the 20th centure were analog computers, not digital.
At some point analog died however, and "computer" basically by default started meaning just "digital computer".
As of the 2010's and forward, with the limit of Moore's law and the rise of machine learning, people have started looking again into analog computing as a possile way forward. A key insight is that huge floating point precision is not that crucial in many deep learning applications, e.g. many new digital designs have tried 16-bit floating point as opposed to the more traditional 32-bit minium. Some papers are even looking into 8-bit: dl.acm.org/doi/10.5555/3327757.3327866
As an example, the Lightmatter company was trying to implement silicon photonics-based matrix multiplication.
A general intuition behind this type of development is that the human brain, the holy grail of machine learning, is itself an analog computer.
Tagged
Tagged
Unsurprisingly the term "computer" became a synonym for this from the 1960s onwards!
Tagged
Homepage: www.phoronix-test-suite.com/
Ubuntu usage examples:
The website went down in 2023, and Ciro Santilli added a data backup to cirosantilli/parsec-benchmark
Ciro Santilli's fork of PARSEC. This fork was made to improve the build system and better support newer targets, including newer Ubuntu and Buildroot.
After the PARSEC website died in 2023, Ciro Santilli also uploaded the test data to GitHub.
The interface is a bit annoying, but the tool is really cool.
100 cycles of
matrixprod:stress-ng -c1 --cpu-ops 100 --cpu-method matrixprodman stress-ng gives the list of possible --cpu-method. It documents matrixprod as:matrix product of two 128 × 128 matrices of double floats. Testing on 64 bit x86 hardware shows that this is provides a good mix of memory, cache and floating point operations and is probably the best CPU method to use to make a CPU run hot.
If you don't specify the
--cpu-method it apparently loops through every method one by one.Limit time to 1s instead of limiting cycles:
stress-ng -c1 -t1 --cpu-method matrixprodThis section is about companies that were primarily started as computer makers.
For companies that make integrated circuits, see also: Section "Semiconductor company".
Tagged
The japanese name literally means:
- 富士 fushi, from Mount Fuji, which itself has unknown origin
- 通 tong: telecommunications
They died so completely, Googling "ICL" now has higher hits such as Imperial College London.
Why the UK's IBM Failed by Asianometry (2022)
Source. Main lesson perhaps: don't put national money to fight already established markets. You have to fight for what is coming up next. E.g. that is part of the reason for TSMC's success.As of the 2020's, a slumbering giant.
But the pre-Internet impact of IBM was insane! Including notably:
- some of the most important business computers of the pre-personal computer era
- SQL
- IBM Generalized Markup Language, which is a predecessor to XML and HTML
Tagged
This is a family of computers. It was a big success. It appears that this was a big unification project of previous architectures. And it also gave software portability guarantees with future systems, since writing software was starting to become as expensive as the hardware itself.
Media:
This was the first major commercial computer hit. Stlil vacuum tube-based.
Learning how to program on the IBM 650 Donald Knuth interview by Web of Stories (2006)
Source. It was decimal!Borrow from the Internet Archive for free: archive.org/details/manbehindmicroc000berl/page/n445/mode/2up
Borrow from the Internet Archive for free: archive.org/details/supermenstory00murr
Initial chapters put good clarity on the formation of the military-industrial complex. Being backed by the military, especially just after World War II, was in itself enough credibility to start and foster a company.
It is funny to see how the first computers were very artisanal, made on a one-off basis.
Amazing how Control Data Corporation raised capital IPO style as a startup without a product. The dude was selling shares at dinner parties in his home.
Very interesting mention on page 70 of how Israel bought CDC's UNIVAC 1103 which Cray contributed greatly to design, and everyone knew that it was to make thermonuclear weapons, since that was what the big American labs like this mention should be added to: en.wikipedia.org/wiki/Nuclear_weapons_and_Israel but that's Extended Protected... the horrors of Wikipedia.
Another interesting insight is how "unintegrated" computers were back then. They were literally building computers out of individual vacuum tubes, then individual semiconducting transistors, a gate at a time. Then things got more and more integrated as time went. That is why the now outdated word "microprocessor" existed. When processors start to fit into a single integrated circuit, they were truly micro compared to the monstrosities that existed previously.
Also, because integration was so weak initially, it was important to more manually consider the length of wire signals had to travel, and try to put components closer together to reduce the critical path to be able to increase clock speeds. These constraints are also of course present in modern computer design, but they were just so much more visible in those days.
The book does unfortunately not give much detail in Crays personal life as mentioned on this book review: www.goodreads.com/review/show/1277733185?book_show_action=true. His childhood section is brief, and his wedding is described in one paragraph, and divorce in one sentence. Part of this is because he was very private about his family most likely note how Wikipedia had missed his first wedding, and likely misattribute children to the second wedding; en.wikipedia.org/wiki/Talk:Seymour_Cray section "Weddings and Children".
Crays work philosophy is is highlighted many times in the book, and it is something worthy to have in mind:
- if a design is not working, start from scratch
- don't be the very first pioneer of a technology, let others work out the problems for you first, and then come second and win
Cray's final downfall was when he opted to try to use a promising but hard to work with material gallium arsenide instead of silicon as his way to try and speed up computers, see also: gallium arsenide vs silicon. Also, he went against the extremely current of the late 80's early 90's pointing rather towards using massively parallel systems based on silicon off-the-shelf Intel processors, a current that had DARPA support, and which by far the path that won very dramatically as of 2020, see: Intel supercomputer market share.
Tagged
Tagged
Tagged
www.threekit.com/blog/gltf-everything-you-need-to-know comparision of several formats
Official demos: github.com/KhronosGroup/glTF-Sample-Assets These are visible at: github.khronos.org/glTF-Sample-Viewer-Release/ with a JavaScript viewer present at: github.com/KhronosGroup/glTF-Sample-Viewer TODO can you load models on the web?
Supports animations, e.g.:
gltf-viewer.donmccurdy.com/ is based on doesn't work with those examples because they have separate asset files.
f3d just worked for it.
Tagged
To test it, let's get two computers on the same local area network, e.g. connected to Wi-Fi on the same home modem router.
On computer B:
- find computer IP with the
ipCLI tool. Suppose it is 192.168.1.102 - then run Ciro's
ncHTTP test server
On computer A, run on terminal 1:
sudo tcpdump ip src 192.168.1.102 or dst 192.168.1.102Then on terminal 2 make a test request:
curl 192.168.1.102:8000Output on terminal 1:TODO understand them all! Possibly correlate with Wireshark, or use
17:14:22.017001 IP ciro-p14s.55798 > 192.168.1.102.8000: Flags [S], seq 2563867413, win 64240, options [mss 1460,sackOK,TS val 303966323 ecr 0,nop,wscale 7], length 0
17:14:22.073957 IP 192.168.1.102.8000 > ciro-p14s.55798: Flags [S.], seq 1371418143, ack 2563867414, win 65160, options [mss 1460,sackOK,TS val 171832817 ecr 303966323,nop,wscale 7], length 0
17:14:22.074002 IP ciro-p14s.55798 > 192.168.1.102.8000: Flags [.], ack 1, win 502, options [nop,nop,TS val 303966380 ecr 171832817], length 0
17:14:22.074195 IP ciro-p14s.55798 > 192.168.1.102.8000: Flags [P.], seq 1:82, ack 1, win 502, options [nop,nop,TS val 303966380 ecr 171832817], length 81
17:14:22.076710 IP 192.168.1.102.8000 > ciro-p14s.55798: Flags [P.], seq 1:80, ack 1, win 510, options [nop,nop,TS val 171832821 ecr 303966380], length 79
17:14:22.076710 IP 192.168.1.102.8000 > ciro-p14s.55798: Flags [.], ack 82, win 510, options [nop,nop,TS val 171832821 ecr 303966380], length 0
17:14:22.076727 IP ciro-p14s.55798 > 192.168.1.102.8000: Flags [.], ack 80, win 502, options [nop,nop,TS val 303966383 ecr 171832821], length 0
17:14:22.077006 IP ciro-p14s.55798 > 192.168.1.102.8000: Flags [F.], seq 82, ack 80, win 502, options [nop,nop,TS val 303966383 ecr 171832821], length 0
17:14:22.077564 IP 192.168.1.102.8000 > ciro-p14s.55798: Flags [F.], seq 80, ack 82, win 510, options [nop,nop,TS val 171832821 ecr 303966380], length 0
17:14:22.077578 IP ciro-p14s.55798 > 192.168.1.102.8000: Flags [.], ack 81, win 502, options [nop,nop,TS val 303966384 ecr 171832821], length 0
17:14:22.079429 IP 192.168.1.102.8000 > ciro-p14s.55798: Flags [.], ack 83, win 510, options [nop,nop,TS val 171832824 ecr 303966383], length 0-A option to dump content.Amazing tool that captures packets and disassembles them. Allows you to click an interactive tree that represents Ethernet, TCP/IP and application layer like HTTP.
Start capture immediately from CLI, capture packets to/from 192.168.1.102:
sudo wireshark -f 'host 192.168.1.102' -kCapture by instead:
sudo wireshark -f http -k
sudo wireshark -f icmp -kFilter by both protocol and host:
sudo wireshark -f 'host 192.168.1.102 and icmp' -kFor application layer capture filtering, the best you can do is by port:There is an
sudo wireshark -f 'tcp port 80'http filter but only for as a wireshark display filterSample usage:This produces simple one liners for each request.
sudo tshark -f 'host 192.168.1.102What you likely want is the
-V option which fully disassembles each frame much as you can do in the GUI Wireshark:sudo tshark -V -f 'host 192.168.1.102TODO didn't manage to get it working with TP Link ARCHER VR2800 even though it shows DHCP as enabled and it also shows MAC addresses and corresponding hostnames in the router management interface.
For IP-level communication, askubuntu.com/questions/22835/how-to-network-two-ubuntu-computers-using-ethernet-without-a-router/116680#116680 just worked between P51 and P14s both on Ubuntu 23.10 connected with a regular Cat 5e cable.
On both machines, first we found the Ethernet cable interface name with the which outputs on the P41s:so the interface was
ip CLI tool:ip a1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp1s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether fc:5c:ee:24:fb:b4 brd ff:ff:ff:ff:ff:ff
3: wlp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 04:7b:cb:cc:1b:10 brd ff:ff:ff:ff:ff:ff
inet 192.168.1.123/24 brd 192.168.1.255 scope global dynamic noprefixroute wlp2s0
valid_lft 61284sec preferred_lft 61284sec
inet6 fe80::3597:15d8:74ff:e112/64 scope link noprefixroute
valid_lft forever preferred_lft foreverenp1s0f0, because wlp is wireless and lo is localhost.So on the P14s we assign an IP of 10.0.0.10 to the P51:
sudo ip address add 10.0.0.10/24 dev enp1s0f0Then on the P51 analogously, giving IP of 10.0.0.20 to the P14s:
sudo ip address add 10.0.0.20/24 dev enp0s31f6And after that, P14s can:and P51 can:
ping 10.0.0.10ping 10.0.0.20TODO after a few seconds, the settings appear to be forgotten, and
ping stops working unless you do sudo ip address add on the local machine again. This seems to happen after a popup appears saying "Activation of network connection failed" as it fails to obtain Internet from the cable.TODO list and delete such manual assignments we've made.
E.g. to give Compueter 2 Internet in a setup like:Can be tested e.g. by turning off Wi-Fi from Computer 2 if it has one.
Internet --- Wi-Fi --- Computer 1 --- Ethernet --- Computer 2Got it working: askubuntu.com/questions/3063/share-wireless-connection-with-wired-ethernet-port/1502850#1502850
This one is not generally seen by software, which mostly operates starting from OSI layer 2.
A good project to see UARTs at work in all their beauty is to connect two Raspberry Pis via UART, and then:
- type in one and see characters appear in the other: scribles.net/setting-up-uart-serial-communication-between-raspberry-pis/
- send data via a script: raspberrypi.stackexchange.com/questions/29027/how-should-i-properly-communicate-2-raspberry-pi-via-uart
Part of the beauty of this is that you can just connect both boards directly manually with a few wire-to-wire connections with simple jump wire. Its simplicity is just quite refreshing. Sure, you could do something like that for any physical layer link presumably...
Remember that you can only have one GNU screen connected at a time or else they will mess each other up: unix.stackexchange.com/questions/93892/why-is-screen-is-terminating-without-root/367549#367549
On Ubuntu 22.04 you can screen without sudo by adding yourself to the
dialout group with:sudo usermod -a -G dialout $USERWhen non-specialists say "Ethernet cable", they usually mean twisted pair for Ethernet over twisted pair.
But of course, this term is much more generic to a more specialized person, since notably fiber optics are also extensively used in Ethernet over fiber.
This is the most common home "ethernet cable" as of 2024. It is essentially ubiquitous. According to the existing Ethernet physical layer, the maximum speed supported is 2.5 Gbit/s.
The frequency range of Wi-Fi, which falls in the microwave range, is likely chosen to allow faster data transfer than say, FM broadcasting, while still being relatively transparent to walls (though not as much).
There is no userland process for it, it is handled directly by the Linux kernel: unix.stackexchange.com/questions/439801/what-linux-process-is-responsible-for-responding-to-pings/768739#768739
Bibliography:
- some good interview excerpts with some of the pioneers on Glory of the Geeks
This is a standard way to embed images in HTML pages with the
img tag.Hardcoded and unique network addresses for every single device on Earth.
Started with 48 bits (6 bytes), usually given as 01:23:45:67:89:AB but people now encouraged to use 64-bit ones.
How they are assigned: www.quora.com/How-are-MAC-addresses-assigned Basically IEEE gives out the 3 first bytes to device manufacturers that register, this is called the organizationally unique identifier, and then each manufacturer keeps their own devices unique.
A "DNS database" is a database that stores DNS records, notably A-records, which IP a domains is hosted at.
For currently live domains, domain to IP can of course be easily determined on the fly by just resolving the domain like the browser does, e.g.
cirosantilli.comWhat is hard however is:
- the other way around is harder however: given an IP, list all domains that it hosts. This is known as "reverse IP" searching.
- historic data, i.e. what was the IP for a given domain at a given date and vice versa
As of 2023, working with DNS data is just going through a mish-mash of closed datasets/expensive APIs.
We really need some open data in that area.
- opendata.stackexchange.com/questions/1951/dataset-of-domain-names
- opendata.stackexchange.com/questions/2110/domain-name-system-record-a-database
- webmasters.stackexchange.com/questions/33395/find-the-ip-address-of-expired-domains/142751#142751
- superuser.com/questions/686195/how-to-find-the-last-ip-used-for-an-expired-domain-name/1793224#1793224
Some links of interest:
- bushart.org/topic/ip
- archive.org/details/internet-mapping
- stackoverflow.com/questions/307553/possible-to-download-entire-whois-database-list-of-registered-domains (deleted question, see archives)
- www.reversedns.ch/en/ has some OK reverse IPs, but you have to do them one by one with CAPTCHA, and we were already past that point when that source was found, so nothing new was found on it yet
- iphistory.net/ announced at www.reddit.com/r/OSINT/comments/1bip8j7/iphistorynet_find_historic_ip_addresses_from/
Bibliography:
- www.reddit.com/r/OSINT/comments/1j8uasm/does_domaintools_offer_historical_reverse_ip_ie/ by Ciro Santilli
- www.reddit.com/r/OSINT/comments/ne27qi/really_historical_whois/
- www.reddit.com/r/dns/comments/1f4y0mg/any_onestopshop_type_sites_that_are_better_for/
- www.arin.net/reference/research/whowas/ you need to request access and they need to approve your usage. Bastards.
They do have historic reverse IP search at dns-history.whoisxmlapi.com/api but their data is not obviously more complete than viewdns.info, e.g.: as of March 2025:Their whois data seems much better quality however.
- 62.22.60.56
- viewdns.info/reverseip/?t=1&host=62.22.60.56 has traveltimenews.com from 2011
- dns-history.whoisxmlapi.com/api is empty
- 66.175.106.158
- viewdns.info/reverseip/?host=66.175.106.158&t=1 has a hit from 2011
- dns-history.whoisxmlapi.com/api is empty
As of 2025, you can do historical whois for free on the API demo under whois-history.whoisxmlapi.com/ but it only shows the 3 newest records.
To unlock that, you have to create an account, which gives you 500 credits, and then:They do normalize Gmail dot trick, but not for the
- tools.whoisxmlapi.com/whois-history-search: each full historical whois report for a domain costs 50 points
- tools.whoisxmlapi.com/reverse-whois-search: each historical reverse whois search search costs 1 point
googlemail trick.TODO is their database amazing?
TODO do they offer historical reverse IP?
assets.applytosupply.digitalmarketplace.service.gov.uk/g-cloud-14/documents/708941/255941323840717-pricing-document-2024-04-17-1304.pdf is an epic document under the gov.uk domain and marked:and marks Iris Investigate as starting at $15k USD for 250 Queries / Month
HIGHLY CONFIDENTIAL - For use only by DomainTools employees and other audiences under NDA only
Acquired by DomainTools in 2021. ChatGPT says it was the most complete DB ever.
Some interesting analysis by Parth Shukla twitter.com/pparth | www.linkedin.com/in/parth-shukla-59583b20/:
Apparently most of the routers were Chinese. No surprise there.
This is the most accessible DNS database online, as it does not require login or payment.
They have reasonable data. It's not fully complete as Ciro Santilli saw on CIA 2010 covert communication websites, but it is very valuable.
Tested as of 2025, they seem to have removed the pre-IP checks on web interface, and just instead use Cloudfare to check that you are human from time to time, which allows for a lot manual searching to be done! Awesome!
Previously, tou could only get about 250 queries on the web interface, then 250 queries per free account via API. They check your IP when you signup, and you can't sign in twice from the same IP. They also state that Tor addresses are blacklisted. They also normalize dots in gmail addresses, so you need more diverse email accounts. But they haven't covered the
.gmail vs .googlemail trick.Their data is also quite disjoint from the data of the 2013 DNS Census. There is some overlap, but clearly their methodology is very different. Some times they slot into one another almost perfectly.
Very curiously, their reverse IP search appears to be somewhat broken, or not to be historic, e.g.We've contacted viewdns.info support and they replied:This is likely not accurate, more precisely it likely only works if it was the last IP address, not necessarily a current one.
- viewdns.info/iphistory/?domain=vuvuzelanews.com hits 74.116.72.246 in 2011, later moved to others
- viewdns.info/reverseip/?host=74.116.72.246&t=1 however does not contain
vuvuzelanews.com
The reverse IP tool will only show a domain if that is it's current IP address.
Data format overview: opendata.stackexchange.com/questions/1951/dataset-of-domain-names/21077#21077
TODO was this data also obtained illegally like the Carna botnet
Tagged
Some interesting usages:
The CIA really likes this registrar, e.g.:
- CIA 2010 covert communication websites
- 2014 www.newsweek.com/former-cia-officials-ready-defend-agency-after-torture-reports-release-290383A group of former CIA officials are gearing up to defend the agency when the Senate releases its long-awaited report investigating "enhanced interrogation" tactics used on prisoners after 9/11. The highlight of their PR push will be a website, "CIASAVEDLIVES.COM," which is set to go live when the report is released on Tuesday, Foreign Policy reported.The domain was registered on November 2 under a private registration name, through DomainsByProxy, a company that guards the identity of registrants.
Some cool ones:
- playinside.me
Archive example: web.archive.org/web/20130726224338/http://librarianhelper.com/
Tagged
Tagged
Polar Music Talks 2013 interview with Martin Lorentzon
. Source. Fun to watch our ADHD friend fidged around in his chair. Also either he's a dwarf, or the interviewer is a giant. Likely a dwarf because his heels don't touch the floor while seated.A network interface controller that does more than just the base OSI model protocols, notably in a programmable way.
- www.nextplatform.com/2022/05/11/intel-unrolls-dpu-roadmap-with-a-two-year-cadence/
- www.trentonsystems.com/blog/what-is-a-smartnic
- blogs.nvidia.com/blog/2021/10/29/what-is-a-smartnic/ "Some are using FPGAs which promise flexibility"
- www.servethehome.com/intel-ipu-exotic-answer-to-industry-dpu/ "Intel IPU is an Exotic Answer to the Industry DPU"
- 2022 www.datacenterdynamics.com/en/news/amd-to-buy-smartnic-firm-pensando-for-19-billion/ "AMD to buy SmartNIC firm Pensando for $1.9 billion"
- www.theregister.com/2022/06/14/alibaba_dpu_cloud/ mentions that Alibaba Cloud created their own.
Hyperscalers Lead The Way To The Future With SmartNICs by The Next Platform (2019)
Source. - youtu.be/kwroXmFJJf0?t=599 financial industry is one of the users, notably high-frequency trading
Associated article: www.nextplatform.com/2019/10/31/hypercalers-lead-the-way-to-the-future-with-smartnics/ mentions that:
Google is widely believed to be working on its own design.
This section explains a tutorial on how to allow external computers to access a computer in your network.
For example, this could for example be used to allow you to access SSH or a web server running on a computer at your home from your laptop when you are outside of your home.
What you need to do seems to be:From there on, you just need to find your public IP and use that, e.g. you could test accessing a test server from your cell phone cellular network after turning Wifi off for the cell phone.
- NAT Forwarding: allows you to select which external port you want to forward to which internal IP + port
- Network > LAN Settings > Address reservation > and reserve the IP above to the MAC of your computer
Ciro Santilli tested this on his TP-Link Archer VR2800 router Virgin Media Hub 3.0 Wifi setup and it just worked. Virgin Media doesn't seem to pose any strict restrictions to this.
A next good step if you are going to have a workhorse computer without much personal logins of value is to setup "group isolation" so that if that computer ever gets compromised, hackers won't be able to infiltrate the rest of the network. But TODO: couldn't find the setting on the TP-Link Archer VR2800 even though the manual says it should be there. Oh well.
Put Hub 3.0 in modem mode on 192.168.0.1. Turn it off. You MUST TURN IT OFF NOW.
TP Link Archer VR2800 192.168.1.1 and set:
- "Operation mode" > "Wireless router mode" (was "DSL Modem/Router mode" by default).
- "Network" > "Internet" > "Add" > "Internet Connection Type" > "Dynamic IP" > "Save"
Custom configs we had, not sure if mandatory:
- Dynamic DHPC mode
- Unicast DHCP
Wait for TP link to fully reboot.
Connect port 4 of tp link (marked WAN/LAn) to port 1 of VM Hub (unmarked, but it is magic, has to be port 1).
Finally, AFTER everything else is setup, turn on the Hub and wait for a few minutes. It ONY WORKS if you turn it on after everything is setup.
Outcome:
- hub light turns purple: www.reddit.com/r/VirginMedia/comments/c703t6/purple_light_on_the_box/
- Archer WAN light turns on white. Not red. Red means error
- you have Wifi. Notably, the 5G Wifi is way way faster and reaches the WAN limit of 256 Mbps.
- Ethernet does not work anymore on either Hub nor Archer, Wifi only. But it doesn't matter because the 5G Wifi already reaches the speed limit.
How to use a TP-Link ARCHER VR2800 modem/router with Virgin Media Hub 3.0 for Wifi
. Source. Bibliography:
- community.virginmedia.com/t5/Forum-Archive/Connecting-Tp-link-archer-vr2800-to-Hub-3/td-p/4765927 This was The thread, the only one that clearly explained the fundamental importance of turn on off ordering by "jbrennand".
- community.tp-link.com/en/home/forum/topic/269540
- community.tp-link.com/en/home/forum/topic/170344
- community.virginmedia.com/t5/Gaming-Support/Connecting-Archer-VR2800-to-Hub-4/td-p/5246513
A switch is a box with a bunch of Ethernet wires coming into it:Except that it doesn't have to be Ethernet, e.g. it would also be a Wi-Fi.
+--------------------+
| +-+ +-+ +-+ +-+ |
| |1| |2| |3| |4| |
| +-+ +-+ +-+ +-+ |
+--------------------+What the switch does is:After the destination is found, a confirmation is somehow sent back to the switch, which then learns which wire to send each MAC address to.
- an Ethernet request came in from wire 1
- decide which wire to send it out on, e.g. wire 2, 3, 4, 5, etc. You likely don't want to send it back through 1 where it came from.
A switch is a bit like a router but it is a bit dumber/operates at a lower level: it basically operates only on MAC addresses, not on IP addresses.
The Internet service provider boxes most people have at home combines a switch for the local network and a router for the ISP communication.
Nerds 2.0.1 excerpt about Cisco (1998)
Source. - youtu.be/mhz24AR3nIc?t=45 the founders both worked at Stanford University but because they were in different departments they couldn't send an email to one another.
- youtu.be/mhz24AR3nIc?t=54 Sandy Lerner is very nice and chilled. She says how she was amazed by Leonard's manners!
- youtu.be/mhz24AR3nIc?t=86 "sincerity begins at a little over 100 hours a week". The dude is a robot.
- youtu.be/mhz24AR3nIc?t=279 earthquake!!!
- youtu.be/d0ya8DggDYs?list=PLn7AqqWS1I_9EHEHy6sw-v6hUMhbeOTRW&t=3268 she bought a manor house, probably in Chawton Hampshire, England, possibly Chawton House
- youtu.be/d0ya8DggDYs?list=PLn7AqqWS1I_9EHEHy6sw-v6hUMhbeOTRW&t=3312 he started donating to search for extraterrestrial intelligence
Nothing phenomenally new on the early days to add on top of Video 14. "Nerds 2.0.1 excerpt about Cisco (1998)", but a few new good points:
- Cisco at one point became the largest company by market capitalization. This wore off a bit as of 2020.They used this overvalued stock in part to buy many other (often also overvalued) up and coming companies. This acquisition spree strategy was apparently not the norm at the time. rohitnair.wordpress.com/2011/12/13/cisco-history-cisco-systems-history-and-trivia-brand-history-and-trivia/ mentions they have bought more than 140 companies since, and that they call this strategy "Build, Buy And Partner"
- a big part of what Cisco did was to allow cheap local communication in-campus. At that time, the ARPANET was already up and running, but their "routers", called Interface Message Processors were very expensive at about $100,000, and to send data across the campus you had to go through them, which meant expensive bandwidth. The routers sometimes failed, and the fallback was to send students around with disks: "sneakernet". They needed new local protocols and hardware to efficiently connect different campus networks.
- Sandy Lerner nude photo
- Cisco was a pioneer in having an Internet support forum. Customers could also help one another. This was fundamental in scaling support, as they grew so fast it would be impossible to hire a support team large enough without the help of the forum.
- Cisco gave out source code to some customers who would then implement protocols they cared about, and Cisco would then merge it back
This chick is hardcore.
She posed naked on horseback for Forbes to promote animal rights in 1997.
A ultra low resolution reproduction of the image can be found at: rohitnair.wordpress.com/2011/12/13/cisco-history-cisco-systems-history-and-trivia-brand-history-and-trivia/
She's kind of lying on top of the horse's back, and you can't see much, just some tastefully light erotica. It's not like she's fucking the horse or anything.
As per stackoverflow.com/a/52351480/895245 our standard test setup is:
while true; do
resp=$"$(date): hello\n"
len="$(printf '%s' "$resp" | wc -c)"
printf "HTTP/1.1 200 OK\r\nContent-Length: $len\r\n\r\n${resp}\n" | nc -Nl 8000
doneIt runs one instance of the Linux kernel and has one IP address. Each node is therefore a complete computer. As such is must also contain RAM memory, disk storage and a network interface controller.
E.g.:
HP Blade Server by Brian Kirsch (2013)
Source. Featuring an HP DL380 blade server, presumably an older model of this series: buy.hpe.com/uk/en/servers/proliant-dl-servers/proliant-dl300-servers/proliant-dl380-server/hpe-proliant-dl380-gen10-server/p/1010026818.
In the video we can see that it contains RAM, disk storage, we are told about two CPUs, and networking interfaces, so it is a complete computer on its own. He also explains that unlike typical rack servers, each blade unit does not have its own coolers and power supply related hardware, which goes instead on the chassis.
Server run on the current machine. That's how all websites are developed and born!
Some people from them contacted Ciro Santilli after Ciro's initial publishing of CIA 2010 covert communication websites.
After a quick Discord chat with them, it was apparent that these people were really cool and knowledgeable.
Also many of them seem to think university is broken and just go hack straigh away.
A perfect example of a dojo learning model.
Also they don't seem to need sleep. Go figure!
With pepole like this, there's hope for Brazil: Section "What poor countries have to do to get richer".
A series of online talks organized by the Boitatech people
Tagged
Ciro Santilli has become slightly obsessed with this story, and the main mastermind Ross Ulbricht.
Ross Ulbricht's open laptop shortly after his arrest at the Francisco Public Library
. Source. He was running some GNOME based distro, could be Ubuntu from that photo, and likely is given that Ross once recommended Ubuntu to his flatmate.The best article available so far is: www.theregister.co.uk/2019/01/29/how_i_caught_silk_road_mastermind (archive) which summarizes what one of the investigators said in a 2019 French computer security conference.
The key living posts are:
- stackoverflow.com/questions/15445285/how-can-i-connect-to-a-tor-hidden-service-using-curl-in-php (archive) which was originally asked under the real name, and then the username was changed to "Frosty", which matches one of the server's logins after the laptop was captured
- altoid early Silk Road mention: bitcointalk.org/?topic=175.70;wap2 (archive)
The big question is of course how libertarian free market ideologically motivated the website was, and how purely criminal greed it was.
The magnitude of the early operational security mistakes does make Ciro think that Ross did it "because he could" and "for the lolz" in a real world Breaking Bad way.
The entry in Ross' diary does resonate a lot with Ciro and any entrepreneur, full diary at: www.wired.com/2015/01/heres-secret-silk-road-journal-laptop-ross-ulbricht/ (archive).
[i]n 2011," [I believe I will be] "creating a year of prosperity and power beyond what I have ever experienced before,Silk Road is going to become a phenomenon and at least one person will tell me about it, unknowing that I was its creator."
Having this kind of feeling, is the greatest thing any human can have, and what motivates all great things.
Capitalizing in illegal things though is a cheat, big things take longer than a few years to reach, but reaching them is that much more satisfying as well.
Other interesting quotes:which Ciro also feels, see don't be a pussy, and:
I hated working for someone else and trading my time for money with no investment in myself.
Everyone knows I am working on a bitcoin exchange. I always thought honesty was the best policy and now I didn't know what to do. I should have just told everyone I am a freelance programmer or something, but I had to tell half truths. It felt wrong to lie completely so I tried to tell the truth without revealing the bad part, but now I am in a jam. Everyone knows too much. Dammit.
Also very worth reading is the San Francisco flat mate account: www.vice.com/en_us/article/ae3q8g/my-roommate-the-darknet-drug-lord (archive).
The murder for hire allegations are also interesting: mashable.com/2013/10/03/silk-road-hits, he paid 80k dollars to undercover DEA agents!
Except for the fact that Ross was an 80 million Dollar drug lord, those accounts sound exactly like what you would expect from any other nerdy startup founder! The:
- "just do it" strategy effectively going to a minimal viable product (manual transaction management!), while making many mistakes along the way, including hiring mistakes and successes when scaling is needed
- the hardship of self bootstrapping your own social network (here with some kilos of mushrooms)
- the variety of periods, from relatively calm, to hair pulling stress during big changes
It is also amusing to see very concretely the obvious fact that the FBI can get a subpoena for all accounts you ever had, e.g. they knew his laptop model from Amazon and brought a corresponding power cable to the arrest! If you are going to be a cyber criminal, don't use your real name, ever!
Should justice be blind? Maybe. But it does hurt for mere non-blind men to see it sometimes. Especially when drug liberalization is involved.
One Mistake Took Down a 29-Yr-Old Dark Web Drug Lord by Newsthink (2022)
Source. Wow nice video, covers most of the interesting annecdotes and the (alledged) investigation procedure.The article that put Silk Road on the public map!
A very good information dump! Notably it has a very timeline.
We need a mini-series, this just doesn't have enough time. Notably, too much focus on dob, and not enough on the development of Silk Road iteslf. Though it is cool to see what his motivations might have been like. One wonders how realistic the character is. Though him meeting Ross Ulbricht personally sounds exceptionally unlikely.
Ross Ulbricht's LinkedIn profile picture
. Source. Still up as of 2023: www.linkedin.com/in/rossulbricht/Ross Ulbricht told her about Silk Road, but then later said he had sold the website to someone else, without really doing that.
- LinkedIn profile: www.linkedin.com/in/julia-vie-35296959/
- Twitter profile: twitter.com/destenyrangel
As of 2023, she's apparently a photographer with a porny focus (boudoir photography).
Some sources:
He testified against Ross: arstechnica.com/tech-policy/2015/01/friend-who-helped-ulbricht-build-silk-road-testifies-against-him/
youtu.be/HBTYVVUBAGs?t=178 claims to contain a photo of him without source. He has no easy to find online profiles.
Variety Jones: IRL, is there anyone with a clue at all? A girlfriend, boyfriend, buddy you talk to, etc?myself: Unfortunately yes, there are two but they think I sold the site and got out, and they're quite convinced of it.
His email was published: baronsyntax@gmail.com. They actaully talked on Google Chat. OMG.
Caroline Ulbricht is the wife of Ross Ulbricht, founder of the Silk Road website which allowed users to buy and sell illegal drugs online. They reportedly met while he was in jail, but it is unclear how they met TODO source.
Caroline's maiden name is very likely "Caroline Debrion" as per stalking of her online accounts below. Her middle name might start with an L giving "Caroline L. Debrion", but this is less certain.
Likely accounts:
- www.linkedin.com/in/carolineld/. Named "Caroline Ulbricht" as of February 2025. Her last employement was as a remote Senior Tech Project Manager / Scrum Master at SiriusXM, a podcast technology company where she was until June 2024. From her feed www.linkedin.com/in/carolineld/recent-activity/all/ she seems to have an interest in computer programming and deeptech which is kind of cool. Many of her posts link to x.com/CrlneD, another possible account but currently suspended. This profile has a 3 archives from 2014 e.g. web.archive.org/web/20140408111206/https://twitter.com/CrlneD under name "Caroline Debrion". Googling finds: cryptonews.com/fr/exclusives/saga-ross-ulbricht-et-silk-road-scandales-derriere-la-peine-3271/ specificlly describing her in French as "voluntary and projct manager for the website FreeRoss", so that must be her maiden name. "ld" from the LinkedIn username must be her middle name starting with L plus Debrion.
- x.com/CrlneD see LinkedIn discussion above. Searching Twitter has many hits on now deleted discdussions related to Ross: x.com/search?q=%22CrlneD%22&src=typed_query. She shas a bit of French stuff going on, maybe she actually has cultural links to France as indicated by her name.
- www.quora.com/profile/Caroline-Debrion. The archive: archive.ph/VEMHR has a better matching profile image. This is further confirmed by this mispost under the wrong Twitter account: x.com/ClemencyForRoss/status/471408997082603520, linking to one of her Quora answers.
Possibly related profiles but uncertain:
- www.tiktok.com/@carolinel.debrion has an account of a Caroline L. Debrion, with the likely L. middle name.
- github.com/0xbzho/onename/blob/6bb8799767967236a7815b8ee469466639d60b48/c/a/caro.json#L14 a OneName account, a Bitcoin-based identity system linking to her twitter "crlned"
- jira.atlassian.com/secure/ViewProfile.jspa?name=1eba64e21192
- fr.pinterest.com/brahms42/
- www.facebook.com/caroline.ulbricht.3/
Ciro Santilli first found her name mentioned without source at: en.m.wikipedia.org/wiki/File:Ross_Ulbricht_Released_from_Prison_(cropped).jpg. Her first name "Caroline" was given by Ross at: x.com/RealRossU/status/1888299042146578925 on February 8th 2025.
As of 2025 most Google hits currently give "Rebecca Caroline Ulbricht Ferreira" however, an unrelated Brazilian molecular biology researcher from UNICAMP. So a good Google search is
"caroline ulbricht" -Rebecca to remove the overlap. This leads to www.linkedin.com/in/carolineld/ hits.x.com/pete_rizzo_/status/1882489065167356024 has a photo of them hugging at the jail parking lot just after he was released in 2025.
Ross Ulbricht with his wife soon after he was released from jail in 2025 in front of a mural
. Source. He still seems to be in jail clothes?
Ross Ulbricht with his wife soon after he was released from jail in 2025 on the road
. Source. 
Ross Ulbricht's wife LinkedIn profile picture according to facecheck.id
. 
Ross Ulbricht's wife's wife credly.com profile picture according to facecheck.id
. 
Ross Ulbricht's wife's wife meetup.com profile picture according to facecheck.id
. 2023 Silk Road's Second-in-Command Gets 20 Years in Prison www.wired.com/story/silk-road-variety-jones-sentencing/
2016 "Exclusive: Our Thai prison interview with the alleged top advisor to Silk Road" arstechnica.com/tech-policy/2016/09/exclusive-our-thai-prison-interview-with-an-alleged-top-advisor-to-silk-road/
2015 The Variety Show On the trail of the man believed to be Variety Jones, one of the architects of the defunct drug marketplace Silk Road. www.vice.com/en/article/wnx5qn/the-variety-show
www.justice.gov/usao-sdny/file/797251/download some kind of case file of his trial.
Bibliography:
- darknetlive.com/post/variety-jones-trivia-f5bf87ce Variety Jones Trivia
- antilop.cc/sr/
The curious thing about VJ is that he actually has some culture and says cool things, e.g.:
IRL - is there anyone with a clue at all? Girlfriend, boyfriend, bunny you talk to, online buddy's who you've know for years? Gramma, priest, rabbi, stripper?
Full name Carl Mark Force IV, the fourth! As mentioned at www.vice.com/en/article/vv7dgj/great-moments-in-shaun-bridges-a-corrupt-silk-road-investigator (this made Ciro Santilli laugh quite hard:
Carl Mark Force IV - the other corrupt cop charged alongside Bridges - is pretty hard to beat, just name-wise.
This is a good thing. It basically contains an entire website, with HTML and assets inside a single ZIP, and a little bit of metadata.
It is incomprehensible why browsers don't just implement it as they already have all the web part, and also ZIP stuff:
The situation is so sad. Ubuntu 21.04 doesn't come with a reader installed by default:
Extract certain pages of a PDF:
pdftk input.pdf cat 2-4 output out1.pdfSometimes Ciro Santilli says half jokingly that user interface does not matter.
This software circa 2010-2020 makes that joke not be funny.
How can a UI feel so clunky!
The most aggravating thing is that it is not immediately obvious why it feels so bad.
This is generally good, especially compared to how crappy Evince, the default Ubuntu one, has been around 2014-2020.
Tagged
The opposite of a human-readable format.
Less evil are BLOBs that come from Reproducible builds.
Invented for Bitcoin.
So dominant that it is usualy called just "zip".
You select a sample noise-only area, and it remove noise from the entire video for you:
In simple terms, represents keystrokes of a piano, but it can likely also represent other effects (TODO confirm: bend, vibrato, slides, attack strength)
Can contain multiple parallel tracks as seen from the Wikipedia example: en.wikipedia.org/wiki/File:MIDI_sample.mid
You can see what it contains well with GUI music editor.
MIDI is fun. It is a basic high level representation of most instrumental music, including beats.
To actually listen to MIDI, you need a software synthesizer, which knows what sound to make for each note. One way to specify such instruments is the SoundFont file format.
Convert MIDI to more common standalone formats:
Contains instructions on how to synthesize MIDI.
Extension: sf2.
Can be imported for example by:
- vmpk
- Csound with
sfload: www.csounds.com/manual/html/sfload.html
Opens a virtual MIDI piano GUI. It just works on Ubuntu 20.04: askubuntu.com/questions/34391/virtual-midi-piano-keyboard-setup/1298026#1298026
VMPK is a virtual device that replicates what you would get by connecting a physical MIDI keyboard to your computer. It is not a software synthesizer on its own. But it does connect to a working synthesizer by default (Sonivox EAS) which makes it produce sounds out-of-the box.
TODO: then I messed with my sound settings, and then it stopped working by default on the default "MIDI Connection" > "MIDI Out Driver" > "Network". But it still works on "SonivoxEAS".
A hello world of actually connecting it to a specific software synthesizer manually on Advanced Linux Sound Architecture with
aconnect can be found at: askubuntu.com/questions/34391/virtual-midi-piano-keyboard-setup/1298026#1298026Save to a MIDI file: askubuntu.com/questions/709673/save-as-midi-when-playing-from-vmpk-qsynth/1298231#1298231
Reasonable default key mappings to keyboard covering 2 octaves.
3 multiple simultaneous keys did not work (tested "ZQI"). This might just be a limitation of my keyboard however.
TODO how to save to a
.mid file? askubuntu.com/questions/709673/save-as-midi-when-playing-from-vmpk-qsynthThe generic tool recommendation question: unix.stackexchange.com/questions/7425/is-there-a-robust-command-line-tool-for-processing-csv-files
Lots of features, but slow because written in Python. A faster version may be csvtools. Also some annoyances like obtuse header handing and missing features like grep + cut in one go: csvgrep and select column in csvkit.
Simple example:output:
printf '00,11,22\n33,44,55\n' | csvgrep -H -c2 -r '^11$' | tail -n+200,11,22More verbose description:
There seems to be no way without a pipe, you seem to need to reparse the columns, e.g. the tutorial at: csvkit.readthedocs.io/en/latest/tutorial/2_examining_the_data.html#csvgrep-find-the-data-you-need does:
csvcut -c county,item_name,total_cost data.csv | csvgrep -c county -m LANCASTERA compiled executable under
/usr/bin/csvtool, has an Ubuntu 23.04 package: manpages.ubuntu.com/manpages/lunar/en/man1/csvtool.1.htmlThere seems to be no sane filtering mechanism however: stackoverflow.com/questions/46540752/using-csvtool-call-to-filter-csv-in-bash
Build failed with
undefined reference to pcre_config on Ubuntu 23.04: github.com/DavyLandman/csvtools/issues/18Unfortunately it is lacking some basic options, like optional header + selecting column by index on
csvgrep (though csvcut has it). The project seems kind of dead.Also unclear if it allows to filter + print only selected columns.
Is this the one?
The fact that you cannot have trailing commans in lists or dicts as in is one of the most infuriating design choices of all time!!!
3, at:{
"asdf": [
1,
2,
3,
]
}Yet another awk-like domain-specific language to do things from the CLI in a ridiculously short humber of character? Oh yes.
echo '[{"a": 1, "b": 2}, {"b": 3}]' | jq '.[] | select(.a) | .a'1Bibliography:
Do you know what is worse than XML? Pseudo XML: stackoverflow.com/questions/5558502/is-html5-valid-xml/39560454#39560454
XML predecessor.
SGML predecessor.
XPath kind of died with the rise of CSS selectors around the beginnning of the 2010's. But that is a shame. XPath is a good standard, and was generally more powerful than CSS selectors for many many years.
Represents image pixel by pixel, rather than by mathematical primitives such as done in vector graphics:
Smaller files, scalable image size, and editability. Why would you use anything else for programmatically generated images?!?!
Companies have been really slow to support SVG features in their browsers, and that is very saddening: medium.com/@michaelmangial1/introduction-to-scalable-vector-graphics-6450c03e8d2e
You can't drop SVG support for
canvas until there's a way to run untrusted JavaScript on the browser!SVG does have some compatibility annoyances, notably SVG fonts. But we should as a society work to standardize and implement a fix those, the benefits of SVG are just too great!
Examples:
- svg/svg.svg a minimal somewhat sane SVG:
- if the
widthandheightproperties were not given, you get the default 300x150, which seems to be set in the SVG standard:
- if the
- how to add na SVG image to a HTML file:
- svg/svg.html: external image. The included file is svg/svg.svg.
- svg/inline.html: inline.
- svg/billion-laughs.svg
- svg/html.svg
- svg/triangle.svg
- svg/viewBox.svg: this attribute allows you to control the default SVG
svg width=andheight=while keeping the coordinates of the drawing untouched. If theviewBoxaspect ratio differs from the width/height ratio, you likely want to play withpreserveAspectRatio, otherwise you would get white spaces by default on the generated image - CSS with SVG:
- svg/style.svg: inline CSS
- svg/style-external.svg: external CSS with:
<?xml-stylesheet type="text/css" href="svg.css" ?>, see also: stackoverflow.com/questions/18434094/how-to-style-svg-with-external-css- svg/subdir/style-external.html: is the relative CSS relative to the HTML or to the SVG? Answer: to the SVG... OMG. So how to make it work reliably?
- svg/current-color.html and svg/current-color.svg: illustrates
fill="currentColor". Only works for inline SVG however... See also: stackoverflow.com/questions/13000682/how-do-i-have-an-svg-image-inherit-colors-from-the-html-document/13002311
- JavaScript with SVG:
- svg/defs.html hows how
defsworks- svg/defs-external.html tries to include external
defsfrom svg/defs.svg, but that fails like everything else related to external SVGs
- svg/defs-external.html tries to include external
This is a pain point as of SVG 1.1...
Examples at svg/background.html which answers from stackoverflow.com/questions/11293026/default-background-color-of-svg-root-element/11293812:
- svg/background-rect.svg
- svg/background-viewport-fill.svg: was part of SVG 1.2, but that whole standard got dropped. Not implemented neither in Chromium 85 nor Firefox 93 as of 2021.
This pain reflects directly on Inkscape: set SVG background color in Inkscape.
The major problem with SVG is text computer fonts. If you make an image with text that depends on one computer font and it is not present in the viewer's machine, it will use some other font, which may overlap with other elements of the image. Some libraries Matplotlib solve this by writing characters as curves, but this produces large files and unsearchable text. The inability of different computer platforms to standardize fonts that must always be present is a major issue.
TODO:
Dropped in favor of SVG 2.
.jpg){kind=link}
A Jacquard loom in action by NationalMuseumsScotland (2016)
Source. The lack of audio is unsettling!PNG reference implementation. Ahh, if feels good to have a dominating open source reference implementation.
Demo of using it by Ciro Santilli: stackoverflow.com/questions/1362945/how-to-decode-a-png-image-to-raw-bytes-from-c-code-with-libpng/36399711#36399711
Tagged
It's not super easy to use at first.
And it sometimes says that the basic drawing thing you want to do is off the project's scope.
But as you learn more about it and further generalize the concepts, there are often reasonable reasons for those design choices.
And the UI looks good :-)
Examples:
- superuser.com/questions/167873/how-do-i-draw-a-box-in-gimp you need to go on a top menu to draw a rectangle
Some answers by Ciro Santilli:
Inkscape is a a good software for editing/creating SVG files.
Its functionality is fundamental for as a software for drawing geometry diagrams, as it is a good middle ground between algorithmic generation, and raster graphics.
At 1.0.2, its UI is a bit terrible:
- the way the menus open on the right with title below the window...
- several defaults are atrocious, e.g. export drawing rather than page
And it crashes from time to time on Ubuntu 21.04. And it has some glaring bugs, e.g.:
But still, it is a very good initiative.
What would be really amazing is if they had constraints: gitlab.com/inkscape/inbox/-/issues/1465 like proper CAD software, it would make it possible to not have to redo entire diagrams when you want to change a small part of them.
There's a tiny little crosshair that you can drag around to set the center of rotation.
And there's a button to make that crosshair snap: inkscape.org/forums/questions/can-a-pivotingtransfrom-crosshair-be-moved-and-made-to-snap-to-a-node-or-a-grid-point/#c14432
This is related to the underlying SVG pain point of SVG background color:
For PNG export:
This seems like a decent option, although it has bugs coming in and out all the time! Also it is quite hard to learn to use.
To get started:
- import a clip
- drag it onto the track area
Shortucts:
- Shift + R: cut tracks at current point. You can then select fragments to move around or delete.
- Shift mouse click drag: select multiple clips: video.stackexchange.com/questions/21598/select-range-of-clips-in-kdenlive
To set the video length, search for "set outpoint" on "monitor".
Add subtitles:then drag on top of the video track. To add only to part of the video, cut it up first.
- Effects
- Dynamic text
Preview has no sound on Ubuntu 20.10. Fixed as of Ubuntu 22.04.
Sound worked on Ubuntu 21.04 though, but it then soon crashed with:
= = SET EFFECT PARAM: "rect" = 0=1188 0 732 242
MUTEX LOCK!!!!!!!!!!!! slotactivateeffect: 1
// // // RESULTING REQUIRED SCENE: 1
Object 0x557293592da0 destroyed while one of its QML signal handlers is in progress.
Most likely the object was deleted synchronously (use QObject::deleteLater() instead), or the application is running a nested event loop.
This behavior is NOT supported!
qrc:/qml/EffectToolBar.qml:80: function() { [native code] }
KilledOn Ubuntu 22.04 haven't crashed yet.
Ubuntu 23.04 broke the clip drag and drop!
Worked on Ubuntu 20.10.
The UI is a bit too buggy to bear.
How to unsplit, can't find on shotcut 21.05.01: forum.shotcut.org/t/is-it-possible-to-unsplit/1466/2
Background noise reduction: couldn't easily find out how, especially with automatic profile detected based on a selected region as mentioned at audacity profile-based background noise removal:
Ubuntu 20.10 crash...:
exceptions:ERROR Unhandled Exception
Traceback (most recent call last):
File "/usr/bin/openshot-qt", line 11, in <module>
load_entry_point('openshot-qt==2.5.1', 'gui_scripts', 'openshot-qt')()
File "/usr/lib/python3/dist-packages/openshot_qt/launch.py", line 97, in main
app = OpenShotApp(argv)
File "/usr/lib/python3/dist-packages/openshot_qt/classes/app.py", line 218, in __init__
from windows.main_window import MainWindow
File "/usr/lib/python3/dist-packages/openshot_qt/windows/main_window.py", line 45, in <module>
from windows.views.timeline_webview import TimelineWebView
File "/usr/lib/python3/dist-packages/openshot_qt/windows/views/timeline_webview.py", line 42, in <module>
from PyQt5.QtWebKitWidgets import QWebView
ImportError: /usr/lib/x86_64-linux-gnu/libQt5Quick.so.5: undefined symbol: _ZN4QRhi10newSamplerEN11QRhiSampler6FilterES1_S1_NS0_11AddressModeES2_, version Qt_5_PRIVATE_APIOn Ubuntu 20.10, just:
ffmpeg -i input.mp4 -vf "subtitles=subtitle.srt" output.mp4To change font size: stackoverflow.com/questions/21363334/how-to-add-font-size-in-subtitles-in-ffmpeg-video-filterThe default appears to be 24, so just multiply that by whatever seems like a reasonable factor.
ffmpeg -i input.mp4 -vf "subtitles=subtitle.srt:force_style='Fontsize=64'" output.mp4Note howver that .ass subtitle files can contain style information, which ffmpeg respects. Aegisub can produce and preview such styles, making .ass one of the best options.
Written in C#.
This worked well on 3.2.2 Ubuntu 20.10. Recommended.
First import video with:They don't have an
aegisub-3.2 ourbigbook-parent.mkvaegisub executable without the version number. Amazing.If you already have a subtitle file that you want to edit, then just pass it on as well:
aegisub-3.2 ourbigbook-parent.mkv ourbigbook-parent.assCtrl + P: play and pause video.
Ctrl + 3: set current substitle start time.
Ctrl + 4: set current substitle end time.
Enter: finish editing the current entry and start a new one.
Good shortcuts and user experience.
No waveform viewer: github.com/otsaloma/gaupol/issues/49 so unusable.
Not to be confused with Subtitle Edit.
As of 0.54.0 this feels featureful, but extremely buggy or lacking UI obvious enhancements that would be simple to implement, and offer huge value:
- fundamental not defined out of box, e.g. Tiny forward/backwards. You can define them yourself, but they should be provided.
- github.com/SubtitleEdit/subtitleedit/issues/4976 can't export SubRip?
- can't interact with video on waveform?
It is hard to understand how that project reached this weird featureful but crappy state. Feels like they just gave push permission to a bunch of random people.
A Microsoft format for flashing microcontrollers by copying files to a magic filesystem mounted on host, e.g. as done on the Micro Bit and Raspberry Pi Pico.
Revolutionary for its time, and a big part of Ciro's Enlightenment.
But too insane, and did not keep up with internet age, and so Ciro wants to kill it now.
Default mathematics typesetting used in OurBigBook Markup.
Key issues:
- github.com/KaTeX/KaTeX/issues/2228
newcommanddid not support optional arguments
LaTeX subset that output nicely to HTML.
Too insane though due to LaTeX roots, better just move to newer HTML-first markups like OurBigBook Markup or markdown.
XML, ain't nobody ever going to write that manually.
The questions are: who is this Mark, and why does he have to go down?
This is good software.
If it only it were written in JavaScript instead of Haskell (!?), then Ciro might have used it as the basis for OurBigBook Markup.
Tagged
Ciro Santilli was contributing to this, when CommonMark left private mode and killed it, thus wasting many hours of Ciro's time.
See also: Ciro Santilli's minor projects.
CommonMark is a good project. But its initial release method was not very nice, they first developed everything behind closed doors with the big adopters like GitHub and Stack Overflow, and only later released the thing read, thus wasting the time of people who were working on alternative in the meanwhile, e.g. github.com/karlcow/markdown-testsuite which Ciro contributed to: Ciro Santilli's minor projects.
These are the rules which specify what different concurrent read/write memory accesses from different threads/processes can or cannot see.
Notable such set of rules include:
- C++ memory model. These are also reflected on the semantics of memory of the corresponding instruction set architecture
- SQL transaction isolation level
That's what usually fucks up parallel programs.
- superuser.com/questions/133082/what-is-the-difference-between-hyper-threading-and-multiple-cores/995858#995858
- stackoverflow.com/questions/680684/what-are-the-differences-between-multi-cpu-multi-core-and-hyper-thread/73405312#73405312
- unix.stackexchange.com/questions/88283/so-what-are-logical-cpu-cores-as-opposed-to-physical-cpu-cores/739296#739296
Hyperthreding is the Intel brand-name, TODO generic name.
It is hard to say if this channel is good because of the awesome information, or if because of the absolute cutness of that British presenter. Maybe it is both.
Ancestors
Incoming links
- Accounts controlled by Ciro Santilli
- Art
- Blade server
- British pragmatism
- Charles Bukowski
- Ciro Santilli's cycling
- Ciro Santilli's musical education
- Classical computer
- Computational physics
- Computer
- Computer science
- Deep tech
- Digital electronic computer
- Emulator
- Good
- Hans Bethe
- Marc Verdiell
- Natural language
- Power engineering
- Programming language
- Richard Feynman
- Scholar-official
- Semiconductor
- Software engineer
- Trader (finance)
- Turing Award
- User interface
- Video game console