If I tell you I have function f with f(a) = 10 and on it's path from a to b, the graph first
increaes by 5 units then by another 10, and then later on drops by 25 units, you can immediately deduce that
f(b) = f(a) + (+5 +10 -25) = 0. The fundamental theorem of calculus uses the same concept:
To see why \int_a^b f(x) dx = F(b) - F(a) with F'(x) = f(x),
we replace f with f' (and hence F with f) and get
\int_a^b f'(x) dx = f(b) - f(a).
Re-arranging terms, we get
f(b) = f(a) + \int_a^b f'(x) dx.
The last line just says: The value of function f at point b is is the value at point a plus the sum of all the infinitely many changes the function goes through on its path from a to b.
As someone who has written his PhD thesis in LaTeX, I wish Markdown, or more specifically Quarto, would have existed at that time. Spent too mich time setting up basic stuff that is now just 2 pre-defines styles away.
I work in a corporate setting and the money and time we wasted because of Zscaler and its SSL inspection [1] is beyond your wildest imagination. Whenever I see a "SSL certificate problem: self-signed certificate in certificate chain" error, I know I'm in trouble.
Zscaler was deployed on a Windows 11 machine at a place that a friend worked at. When I assessed the software, its behavior was downright evil like malware. As we all know, it injects its own root certificate into the operating system in order to conduct man-in-the-middle attacks on TLS/HTTPS connections to monitor the user's web activity.
Furthermore, it locks down the web browser's settings so that you cannot use a proxy server to bypass Zscaler's MITM. I saw this behavior in Mozilla Firefox, where the proxy option is set to "No proxy" and all other options are disabled and grayed out; I imagine that it does the same to Google Chrome. If you try to modify the browser's .ini(?) file for proxy settings, Zscaler immediately overwrites it against your will. Zscaler worked very hard to enforce its settings in order to spy on the computer user.
And as you'd expect, if you open up the Zscaler GUI in the system tray, you are presented with the option to disable the software if you have the IT password. Which of course, you don't have. Then again, that might be an epsilon better than the Cybereason antivirus software, which just has a system tray icon with no exit option, and cannot be killed in Task Manager even if you are a local administrator, and imposes a heavy slowdown if you're open hundreds of small text files per second.
I feel the same way about Cisco Umbrella where I work.
The worst breakage by far is protocol breakage; basically anything that uses HTTP as a basis for building some other protocol gets broken all the time. None of the people implementing it seem aware. They buy the vendor's claim that it's "transparent", when in fact even "inspect/trace-only" modes often break all kinds of shit.
and probably some other things I'm forgetting. When this breakage is reported, the first round of replies is typically "I visited that domain in my enterprise-managed browser and it's not blocked". That is, of course, a useless and irrelevant test.
Often it takes hours to even fully diagnose the breakage with enough confidence to point the finger at that tool and not some other endpoint security tool.
I'm not sure if the people buying and deploying tools in this category don't know how much stuff it breaks or just don't care. But the breakage is everywhere and nobody seems prepared for it.
I really don't understand why people think it's a good idea to use csv. In english settings, the comma can be used as 1000-delimiter in large numbers, e.g. 1,000,000 for on million, in German, the comma is used as decimal place, e.g. 1,50€ for 1 euro and 50 cents. And of course, commas can be used free text fields. Given all that, it is just logical to use tsv instead!
I learned to program at 33 or so (in bioinformatics), my first real lesson a couple of days in: "Never ever use csv". I've never used pd.read_csv() without sep="\t". Idk where csv came from, and who thought it was a good idea. It must have been pre-spreadsheet because a tab will put you in the next cell so tabs can simply never be entered into any table by our biologist colleagues.
I guess it's also why all our fancy (as in tsv++?) file types (like GTF and BED) are all tab (or spaces) based. Those fields often have commas in cells for nested lists etc.
I wish sep="\t" was default and one would have to do pd.read/to_tsv(sep=",") for csv. It would have saved me hours and hours of work and idk cross the 79 chars much less often ;)
Funny story: I once bought and started up Galactic Civilizations 3.
It looked horrible, the textures just wouldn't load no matter what I tried. Finally, on a forum, some other user, presumably also from Europe, noted that you have to use decimal point as a decimal separator (my locale uses a comma). And that solved the problem.
CSV can handle commas in fields just fine (quotes are required in that case). The root problem here is not the format, it's a bug in the CSV exporter used.
It's one of those things where people think, it's there, and it works.
The whole business of software engineering exists in the gap between "it works today on this input" and "it will also work tomorrow and the day after and after we've scaled 10x and rewrote the serialization abstraction and..."
See also: "Glorp 5.7 Turbo one-shot this for me and it works!"
I've embedded large JSON blobs in CSVs. The format is fine and quite robust, just never open it in Excel unless you are prepared for your data to be silently broken, but that's Excel being abhorrent, not CSV: Libreoffice and Google Sheets don't do that.
The JSON in CSVs does piss off the Jetbrains table viewer sometimes though, it will randomly decide to stop parsing a 50k line CSV at halfway through the JSON of line 300ish even though that JSON is no different from other JSON it parsed just fine.
But python reads and writes them fine, as does whatever SQL engine I'm touching, as does other tools.
If you snapped your fingers and removed CSVs from the world your lights would go out within the hour and you'd starve within the week. Trillions of dollars in business are done every day with separated values files and excel computations. The human relationships solve the data issues.
> I really don't understand why people think it's a good idea to use csv.
Because it's easy to understand. Non-technical people understand it. There is tremendous value in that, and that it's underspecified with ugly edge cases doesn't change that.
And you get the under reporting of COVID information in the UK as they passed around CSV files with too many rows for the tools they used.
An interchnage format needs to include information showing that you have all the data - e.g. a hash or the number of rows - or JSON/XML/s-expressions having closing symbols to match the start.
For CSV, I don't know how this comes out. It depends on the library/programming language. It might be 73786976294838210000 or it might throw an exception, or whatever. I'm just saying JSON will not solve your problems neither.
If you need something unambiguously specified, then XML with XSD is still a valid option. All number types are specified exactly, and you can use extensions for custom number types.
There is a saying about Gauss: when another mathematician came to show him a new result, Gauss would remark that he had already worked on it, open a drawer in his desk, and pull out a pile of papers on the same topic.
One of the things I admire about many top mathematicians today like Terence Tao is that they are clearly excellent mentors to a long list of smart graduate students and are able to advance mathematics through their students as well as on their own. You can imagine a half-formed thought Terence Tao has while driving to work becoming a whole dissertation or series of papers if he throws it to the right person to work on.
In contrast, Gauss disliked teaching and also tended to hoard those good ideas until he could go through all the details and publish them in the way he wanted. Which is a little silly, as after a while he was already widely considered the best mathematician in the world and had no need to prove anything to anyone - why not share those half-finished good ideas like Fast Fourier Transforms and let others work on them! One of the best mathematicians who ever lived, but definitely not my favorite role model for how to work.
Well, in that time it was more or less how mathematics worked. It was a way of showing off, and often it would be a case of "Hey I've solved this problem, bet no-one else can". It was only later it became a lot more collaborative (and a bit more focused on publishing proofs).
You're correct that the culture of mathematics has changed a lot, and has become much more collaborative. The rise of the modern doctoral training system in Germany later in the 19th century is also relevant. So really Gauss's example points primarily to how much mathematics has changed. But at the same time, I think you could reasonably take Gauss to task even applying the standards of his own era - compare him with Euler, for example, who was much more open with publication and generous with his time and insights, frequently responding to letters from random people asking him mathematical questions, rather like Tao responding to random comments on his blog (which he does). I admire Euler more, and he was born 70 years before Gauss.
Of course, irascible brilliance and eccentricity has an honorable place in mathematics too - I don't want to exclude anyone. (Think Grigori Perelman and any number of other examples!)
There's also this notion of holding themselves to their own standards.
They, Newton included, would often feel that their work was not good enough, that it was not completed and perfected yet and therefore would be ammunition for conflict and ridicule.
Gauss did not publicize his work on complex numbers because he thought he would attacked for it. To us that may seem weird, but there is no dearth of examples of people who were attacked for their mostly correct ideas.
Deadly or life changing attacks notwithstanding, I can certainly sympathize. There's not in figuring things out, but the process of communicating that can be full of tediousness and drama that one maybe tempted to do without.
Weird typo in what I wrote. It's past the edit window. This is what I had meant to type:
There's joy in figuring things out, but the process of communicating what has been so figured can be tedious and full of drama -- the kind of drama that one maybe tempted to do without.
Someone blew my mind by convincing me to read Bush’s “As we may think” which was published in 1945. Then I started digging into him and discovered he was also the second president of the ACM, was instrumental in shaping the formation of the National Science foundation (mainly by critiquing their initial plans as unworkable) and also Claude Shannon’s doctoral advisor. Because of course he was.
Not to mention instrumental in getting the Manhattan Project going, along with many other research projects during WWII. He basically knew everyone. I didn't know he was Shannon’s advisor though!
> There is a saying about Gauss: when another mathematician came to show him a new result, Gauss would remark that he had already worked on it, open a drawer in his desk, and pull out a pile of papers on the same topic.
As if phd students need more imposter syndrom to deal with. Ona serious side, I wonder what conditions allow such minds to grow. I guess a big part is genetics, but I am curious if the "epi" is relevant and how much.
Imposter syndrome? If I was a PhD-level student (back then) and had an idea - and it turned out that Gauss had also thought of the idea, then written it out, and he kept the notes right in his desk - yeah. I'd take that as proof that I was one of the world's top mathematicians.
Or a good one, forcing governments to have robust infrastructure that this info isn't useful. Similar reasoning as with security and open source software.
Yeah, and it’s not like the enemy would take the information from here, they already have it and likely even more detailed. It is quite basic stuff to have when preparing to defend (or attack).
I'm a data scientist and a lot of my R code are dplyr-chains a la data |> select(features) |> filter(low_quality) |> mutate(feature=...).
It just saves time to comment on what those chains do instead of having go through them every time I want to change something.
or even
$ cat source | <find the flags and give me some documentation on how to use this>
Could you please elaborate on this? Do I get this right that you can set up your your command line so that you can pipe something to a command that sends this something together with a question to an LLM? Or did you just mean that metaphorically? Sorry if this is a stupid question.
Having gone through the explainations of the Transformer Explainer [1], I now have a good intuition for GPT-2. Is there a resource that gives intuition on what changes since then improve things like more conceptually approaching a problem, being better at coding, suggesting next steps if wanted etc? I have a feeling this is a result of more than just increasing transformer blocks, heads, and embedding dimension.
Most improvements like this don't come from the architecture itself, scale aside. It comes down to training, which is a hair away from being black magic.
The exceptions are improvements in context length and inference efficiency, as well as modality support. Those are architectural. But behavioral changes are almost always down to: scale, pretraining data, SFT, RLHF, RLVR.
To see why \int_a^b f(x) dx = F(b) - F(a) with F'(x) = f(x),
we replace f with f' (and hence F with f) and get
\int_a^b f'(x) dx = f(b) - f(a).
Re-arranging terms, we get
f(b) = f(a) + \int_a^b f'(x) dx.
The last line just says: The value of function f at point b is is the value at point a plus the sum of all the infinitely many changes the function goes through on its path from a to b.