I lent Greg a Minivac 601 for a week or so last month. He returned it in a big zippered bag, which I stole because it was pouring rain and I needed to bring the Minivac back to my car.
I was going to return it at the MIT swapfest in a week, but I think he can't make it anymore. Sorry Greg!
Well I'm even more 1000%%%%%% coming to the swapfest now!!!! My precious bag, haha.
Also, you are the most incredibly generous person for lending me that extraordinary piece of computing history!! I recorded its relay sounds and actual resistances. I truly cannot tell you how happy it made me.
I have seen it firsthand in the CS department here at Dartmouth. It is bad.
We're currently designing a new intro systems curriculum, and we're thinking of it as an adversarial problem. That is, we're designing the course to ensure that a student optimizing for the best grade per unit work still meets our learning objectives. That means, as everyone else is saying, paper exams, but also 1-on-1 interviews to check that students understand each assignment they turn in. These interviews feature both factual questions ("You're using this macro from that library. What does it do?", "Please describe what this function does and how it works.") and conceptual questions ("Why is this code structured this way instead of $whatever?", "How else did you try solving this?", etc.) This doesn't stop students from generating code, but at least they have to understand that code in detail.
This is not as good as writing the code yourself, but how much worse is it? For math classes, this gap is gigantic. Obviously, understanding someone else's proof is much easier than writing your own. For programming classes, I think (without evidence) that the gap is somewhat smaller.
My experience from the past is that when this kind of evaluation is made clear up front, the students know what to expect and either do fine or drop the class in the first week. If you start with take-home exams and then spring paper exams on them halfway through the course, then half the class is cheating and won't be able to recover, as we read in the article.
In general, our students are somewhat motivated by an abstract desire to learn, but are much more motivated by grades. If there exists a straightforward path through your course that leads to a good grade without doing much work, most students will take it. (Our undergrads' course review website is literally called "Layup List." They are actually this shameless.) It's our job as instructors to ensure that all paths leading to a good grade either require learning the material or are more difficult to pull off than just learning the material.
It's best not to blame the students. They are good at optimizing metrics; that's how they ended up here in the first place. We just need to better align the evaluation metrics with the outcomes that we're looking for.
There are dozens for every one who didn't get the opportunity to attend an Ivy. The penalty for cheating should be automatic expulsion. I'll note as a hiring manager the college degree won't be worth shit unless the school can show their students have any integrity.
> The penalty for cheating should be automatic expulsion.
Historically, the penalty for cheating at Dartmouth was a 9-month suspension for a first offense (no matter how small, in theory), and permanent "separation from the college" for a second offense. Back in the 90s, there were multiple incidents where this wasn't properly applied to the CS department because the academic committees in charge of punishment were bad at evaluating plagiarism of source code.
Dartmouth certainly should blame the students. Their policy is historically clear on that point. It is the responsibility of professors to use their syllabus to clearly define "cheating" for a particular course, and it is the responsibility of the students not to cheat. The only case where this should be even slightly complicated is if the professor hasn't been sufficiently about what constitutes cheating (and there was one major historic scandal related to this).
I certainly agree that any degree which allows rampant cheating will quickly become a joke to employers.
Employers past a certain scale are not really using degrees as anything other than a low pass filter, and the people judging how qualified hires were in practice are not the ones deciding the minimum requirements, so there's no avenue for feedback on that point.
Our whole education system is setup as a skinner box for grades. People finding tricks for better grades are massively rewarded. Study methods are optimised for recall within a month. And educators are accountable only to the measured outcomes.
Very little of the designed system of education is aimed at teaching. Most of it is aimed at measuring and certifying. Treating students as adversarial grade maximisers will likely teach a lot of them they are expected to grade maximise adversarially.
Yes, unfortunately that is exactly how most education systems are designed. A lot of it is also historical baggage (at least in the European school tradition), where states were faced with the issue of educating the masses, which required a lot of standardization and thus also grades. Although nowadays, educational science has long established the detrimental effects of grades, they are still very widespread. Grades are institutionalized nowadays; you have generations of students who excelled in this Skinner box and became teachers themselves, thus perpetuating the grade box.
Fortunately, there are a few alternatives, schools without grades, that don't focus on short-term recall but long-term understanding, intrinsic motivation, autonomy, and self-actualization, like the Ecole d'Humanité in the Swiss Alps: https://ecole.ch/
You said it well and my brain automatically rephrased your comment while I was reading it:
Our whole everything is setup as a skinner box for mulah. People finding tricks for more mulah are massively rewarded. Efforts are optimized for ROI within a fraction of a second. And management are accountable only to the measured outcomes.
The issue is that education is the only way to get a well paying job. This means that students aren’t there to study the subject, they are there to get to a job.
When further education wasn’t mandatory, things like the honor code made sense.
There are schools with severe penalties for cheating (Army/Navy, Haverford, Davidson), but most aren't set up for that. Professors didn't self-select to be prosecutors, and we didn't spend decades becoming world experts to then become cops.
Love the sinner, hate the sin. If you let yourself start blaming students, you open a pretty corrosive path of moral judgement. Students are clever, and they're making choices informed by what they see in the world. Don't blame them for being unwilling to fail a class that cost a down payment on a house. Don't blame them for seeing powerful people get away with cheating and trying to do the same.
Kallus is right: qualitative assessment is an adversarial problem. Build the assessment correctly, and all of those previous points become moot. By clearly establishing and enforcing the rules you make it more fair to boot.
Totally agree. The moral decay in higher education has a lot to do with it deciding to trade its integrity around cheating by students and professors in exchange for tuition cheques and grant money.
Yes. Best not blame the technology that was specifically built to pass up automatically generated text for human work, or companies that profit from students using it, or government that won't introduce measures that would at least make it possible to comprehensively ban it on campuses.
I doubt it. The concept of cheating is largely unique to academia and a few uptight professions like law. Everywhere else it is just collaboration or learning or simply something nobody cares about.
I would say that tech largely rewards the behaviours people in academia call cheating.
when you cheat on your partner you hurt them by betraying their trust. when you score from offside you can steal a win from the other team thats playing fair. if you lie about your company financials you steal from the government or your creditors.
cheating in schools is nothing like that. there is no direct harm to anyone, financial or emotional. the only real thing is if enough students do it thats bad for the schools reputation and others will treat degrees coming from there as more suspicious.
thats a very abstract and indirect harm, and it depends on employers caring about what school you came from more than a basic "reputable or total scam". to be fair thats pretty common in some industries (it shouldnt be) but i dont think the average student will get worse job offers because some of their classmates cheated.
By cheating you gain a certificate that says you can do things which you cannot. The difference between cheating in university and forging a degree is the time it takes.
When going to a doctor, you wouldn't want them to have cheated for their exams during training. You expect them to be well equipped to practice medicine and to ensure you get correct diagnoses and care. When hiring a programmer or software engineer, you should expect them to be able to provide you a proper, well architected application given the requirements.
The penalty for cheating should be automatic expulsion.
Universities need to stay ahead of how people actually work. Framing the use of AI as 'cheating' gives a university a reputation that it isn't keeping up with industry, which has a wildly negative impact on admissions. You can't expel people for 'working like they do in the real world'.
That's an amusing analogy, but it's conflating university as a purely academic pursuit (learn these things in order to know the things) with university as vocational training (learn these things so you can get a job.) As people have to pay for their degree through loans now, people see it as a means to get the career they want. It's not learning any more. It's training.
So, to continue the analogy, a degree is the equivalent of a forklift truck license, and people do want to drive their forklift truck to the gym. Because they're paying to be able to do that.
If you use AI to complete all your assignments and exams, then getting a degree is less like getting a forklift truck licence (which indicates you've learned some useful skill that few people have) and more like getting a Waymo account (showing that you're competent enough to enter your credit card details and destination into a mobile app).
Undergraduate degrees are not vocational in any of the leading universities in any field. They are sometimes a prerequisite to vocational training, but aren't one in and of themselves.
There's a pretty good reason for that - the base assumption is that training in fundamentals, methods and ways of thinking is something you won't get anywhere else.
> You can't expel people for 'working like they do in the real world'.
You totally can, though. In the real world if I can't remember something, I might look it up in a textbook. Closed-book tests have historically been a totally accepted practice, though, and getting caught bringing notecards with textbook info secretly into a closed-book test would absolutely bring about disciplinary action.
One problem here is that students that use AI to outsource thinking become people who cannot think. These people are not likely to be very useful to employers or even society. We have to figure out how to allow AI to outsource drudgery but not the thinking itself. It should be a better and better bicycle for the mind not a replacement for the brain.
> Obviously, understanding someone else's proof is much easier than writing your own.
Unrelated, but once you get to a more mature level, say grad school and above, I can say this is not always the case! Just like it's easier sometimes to Roll Your Own Damn X in programming, so too are some expositions of proof so dense (necessarily or unnecessarily so) that it is a less taxing affair to simply figure it out yourself, or at least figure out 90% of it, consulting a suggestive sentence or two in the proof in order to get at that last 10%.
Maybe this observation of mine is not so unrelated after all. I don't regret many of the times that I've thrown up my hands at the rococo explanations or solutions given to me by LLMs and simply did my own work. The Socratic method with AI is sometimes more effort than it's worth.
Understanding 90-95% of someone else's proof is much easier than writing your own ;)
It's hard to verify that you understand someone else's work at 100%, but it's reasonable for an expert to tell once they've built it themselves, right?
> It's best not to blame the students. They are good at optimizing metrics; that's how they ended up here in the first place.
As an alumnus of Dartmouth College's CS program, I am sad to hear that my alma mater has sunken so low. Look, I know that the Committee on Students was historically bad at handling CS plagiarism cases back in the 90s (compared to ones in the humanities). But Dartmouth's historic solution to this sort of pernicious "optimization" was to reduce the expected value of cheating by imposing extreme negative consequences on anyone they caught, with a 3-term suspension and a permanent transcript notification for a first offense.
Allowing widespread cheating and LLM regurgitation will destroy a school's reputation with graduate schools and employers, and rightfully so.
You reminded me of my Univ. Math's class (on integrals). The test was basically get one integral equation randomly and solve it in the blackboard only with the teacher present.
Because you didnt know which type of integral you will.get, we had to learn to solve all of them. We made.hundreds of exercises with a small group of close friends (the teacher gave us a book that's infamous in Mexico from an author Schaumm).
> This doesn't stop students from generating code, but at least they have to understand that code in detail.
I mean even before AI this should get tested, as people could just copy code from SO or other websites, from books, or from their classmates.
I've done a few projects where the other students didn't really do much (in hindsight this was probably because I was excited and productive instead of helping and delegating work, which was selfish of me), but they were caught out in 1:1 interviews.
People optimizing for grades is so sad. I got a BS in CS and no one has ever asked about my grades. I tried to optimize for learning and it turns out that pays off really well once you hit the real world.
If you want to be a MD or a JD, then you must optimize for grades. The entrance into post secondary education requires it to be so.
Seeing as the article was about Brown and their PLME program is famous for med school acceptance, then I think we should expect that the culture there is more about grades than a generic undergrad university's culture. Which, yeah, is also very grade focused.
Do you mean very well? Lots of proof out there that it works; Fin, Sierra and others already operate on a value based pricing model where they only get paid if the AI actually resolves issues.
A lot of restaurants in Korea and elsewhere have replaced human cashiers with touch screen kiosks. You’re given a number, someone calls it out when your drink’s ready.
For the places I went to it worked just fine. Did you have a bad experience?
The voice thing didn't work well enough, in Dutch at least. I've done an in-between. We instructed an AI as the client. Then as the teacher, I'd represent the client and would type. Students would form project groups, and each group would get 15-30 minutes to ask questions from the "client".
> the OpenBSD project has not been very receptive in the past for bug reports, my sense of “this is probably fine, in practice”, and that if OpenBSD wants to weed out UB from their code base, then that’s a major project that should be done in a better way than me just being the middle man between the LLM and them for a patch here and there.
Part of the reason for all the UB in OpenBSD is that UBSan doesn't run on that platform. When I ported OpenBSD's httpd to Linux, I found that UBSan tripped before the server even came up because the config flag parsing shifts into the MSB of a signed integer.
I tried to contribute back a patch (just make the flag bitfield unsigned), but it was ignored. I think if UBSan ran natively on OpenBSD, then there would be a lot more of these patches, and the maintainers would have to take an official stance on whether they think these bugs matter.
I hope he ends up covering integer division by constants. The chapter on this in Hacker's Delight is really good but a little dense for casual readers.
It's a custom assembler built on top of the LLVM assembler (llvm-mc) that emits instrumentation code to catch ABI violations at runtime. Stuff like clobbering nonvolatile registers, misaligning the stack pointer, misusing the redzone, assuming volatile registers don't change across a function call, etc.
Hoping to finish up basic x86_64 support within the next few days. I can now reliably assemble and run unoptimized gcc output without hitting false positives, but I still have to iron out some false positives triggered by OpenSSL's handwritten assembly routines.
TODO items for the near future include porting the runtime support library into a kernel module so I can instrument Linux, and beginning ports other architectures (ideally something semi-obscure like POWER or RISC-V). I also need to figure out how to support dynamic linking, because the tool currently needs static linking to access its thread-local variables.
I ran the servers for our networks, binary exploitation, and intro systems programming classes, and this thing is a major annoyance. It is because of this stupid RAT that students do not understand how to use the OpenSSH client.
I've tried a few things to fix this:
1., I set a motd on our class servers asking the students not to use the VSCode remote server plugin.
2. I ran `ncdu /home` in front of the class and demonstrated how, without exception, everyone with >100MB of disk usage on the class server was a VSCode user.
3. I set a user process limit of 45, because the VSCode RAT (somehow) uses ~50 Node processes. When students inevitably ignored the motd and the in-class warning, they hit the process cap and had to ask us to kill their processes in order to get back onto the system.
4. I replaced the process limit with a script that kills all the .vscode-server RATs every 10 seconds.
Your comment is bringing back so many memories of being in university and working around the anachronistically strict limits the uni sysadmins had on the network
Ah yes, one of my favorite skills learned in school was how to work around arbitrary rules made up by cranky sysadmins! I still use it all the time.
The smart kids (whom I’m sure look like they are learning that precious openssh client) are doing their assignment locally or on a free tier VPS with VS Code and scping the thing over when it’s done.
They’re also smart enough to learn openssh when they need it IRL.
In the class which I have managed the infrastructure for, we have 1 to 2 students per VM, so this is less of an issue and there aren't any restrictions.
One of the reasons we provide the VMs is so that students can experience working in a remote server environment. The concern that I have is that these remote ssh tools allow you to bypass learning/practicing how to perform basic actions, e.g. cd, read/edit files.
Granted, as mentioned, you can scp/rsync (or git pull), but at least this seems to be more appropriate when you eventually need to interact with a real production server.
This isn't something specific to VSCode being popular. When I was in college over a decade ago, there were students using Sublime with an SFTP plugin, coding locally and copying stuff with FileZilla or other similar GUI clients to transfer files (I distinctly remember seeing some program that was named something duck-related but for the life of me I can't remember any other specifics). Sublime in particular could be frustrating in the class I TA'd where they had some assignments to deal with processing some (very basic) machine code to simulate running the tool the class used to assemble and run the code from the (similarly basic) ISA they had been taught, which involved them using hexdump or something similar to understand how the bytes in the files worked, but Sublime "helpfully" would render the object files as as their text representation of the hexdump, with extra spacing for readability and swapped endianness compared to the way the bytes would show up if hexdumped on the school's Linux server. There would always be several students every semester who would show up at office hours unable to figure out why their code that was written to try to read an ASCII string like "AD DE EF BE " but instead would instead find some text they didn't recognize because they didn't think to look at the actual byte values, which would always just happen to start with 0xDE, 0xAD, 0xBE, and 0xEF.
The issue is not about live editing on servers, it’s about running a process out on the server that spawns,according to GP, 50-some processes. VS Code remote editing and Sublime/any SFTP/SCP-style deploy are on completely different levels.
I was responding directly to the parent comment's decrying of VS Code as responsible for why their students don't know how to use ssh clients, not the general issue, which I agree is not specifically about live editing.
Off-topic: just one look at that homepage, the icons in particular, brought back such fond memories of the pre-flat design time. Those halcyon days when it was normal for buttons to look like buttons. (Maybe I'm just using the wrong apps these days.)
Can you explain why though? I get that you had to put a lot of effort into blocking VSCode but it's not clear what have VSCode caused to motivate this.
It sounds like they’re operating at larger scales, with a large number of users sharing a resource. Each user spawns 50 processes and consumes 100MB of space if they’re using VSCode. Assuming a low count of 1000 users, that’s 50K running processes and 100GB of space consumed. To enable a text editor.
100GB of storage is… not much for 1000 users. A 1TB NVMe SSD is $60. So 100GB is a total of $6 of storage… or about half a penny per user.
And that’s for SSD storage… an enterprise-grade 14TB hard drive is only $18/TB on Amazon right now, less than a third of the SSD price per TB. Call it 100GB = $2 of storage, total, to enable 1000 users to run the editor of their choice.
So, no, I’m not seeing the problem here.
If you really wanted to penny pinch (one whole penny for every 5 users), I think you could use btrfs deduplication to reduce the storage used.
Very apparently you're counting as a singular physical person. In a big organization there's always an overhead both in time, and money, and more importantly there are more problems to solve than there are resources. So one has to arrange priorities, and just keep low-prio things in check not letting to boil over the lid.
Preventing people from using their preferred tools — tools which are extremely widely used in the real world — does not seem like a useful application of time and effort.
Usefulness depends on conditions of which we don't know a lot here. Sure, there are situations when counteracting pressure is more expensive than expanding capacity. But frankly I doubt this particular case is one of those.
How many concurrent users can you run off a single NVMe SSD?
How many students leave their coursework to the last minute?
How do you explain that the server went down during the last hour before submission deadline again, and that everyone gets an extension again, because you cheaped out on putting the cheapest possible storage into a system that has to cope with large demand at peak times?
How many students now start to do worse because of the anxiety caused by these repeated outages?
How much more needs to be invested in the university counselling services to account for this uptick in students struggling?
No… it’s not. To quote the message earlier in the thread, that message said “everyone with >100MB of disk usage on the class server was a VSCode user.”
100MB * 1000 users is how the person I responded to calculated 100GB, which is storage.
Most of the RAM usage would likely just be executable files that are mmap’d from disk.. not “real” RAM usage. But, also, the 1000 users in question wouldn’t all be connected at the same time… and I honestly doubt they would all be assigned to the same server for practical reasons anyways.
It’s not easy to estimate the real RAM usage with back of the napkin math.
Depending on what they're doing, it could easily be multiple Gb per user. When you do VSCode remoting, pretty much everything but the UI is running on the server. This includes stuff like code analysis for autocompletion, which - especially for languages that require type inference to provide useful completions - can consume a lot of RAM, and a fair bit of CPU.
> I honestly doubt they would all be assigned to the same server for practical reasons anyways.
The computer science department at my university had multiple servers. All CS students got an account on the one same server by default. Access was granted to other servers on a case by case basis, based on very course-specific needs.
So yes, in my case, all CS undergrads used the same one server.
> 1., I set a motd on our class servers asking the students not to use the VSCode remote server plugin. ...When students inevitably ignored the motd...
The provider of the servers? If the students want to use their own server, I’m sure they’re free to do whatever they want. Also VS code has plug-ins that just SFTP the files up.
We encourage students to use their own machines. We even loan out old ThinkPads and MacBooks for students to run Linux on. Very few (3-5 per term) take us up on this offer. Most of our students would rather we do the sysadmin stuff for them.
The only requirement is that the code needs to run on our server, which they can easily check before submission.
You sound like a painful teacher. Who cares if the students are using vscode+remote? Are they learning the material? Are they building good software? This is uni for christ sake, not an apollo mission. The entire purpose of a university environment is to foster learning and development, to open up a safe space for learning. Not to suffocate them with arbitrary and excessive rules.
Things need to be introduced at the right gradient. If you pile it on all at once, nothing is learned. If you want to teach someone how to use SSH to interact with a server, then that should be a dedicated chapter or portion of the curriculum. Once you are beyond that and it comes down to building programs, you need to eliminate that part of the gradient and focus 100% on mastering the task at hand.
My experience participating in Google's program has been pretty good. The reward money is a nice supplement to my grad student stipend. I got a free trip to DEFCON out of it, too.
I was going to return it at the MIT swapfest in a week, but I think he can't make it anymore. Sorry Greg!
reply