It doesnt help that guides ive seen have been pretty handwavy or are not specific enough to the individual situation (i have z hardware, heres how its done). It also doesnt help when every post on HN i see is like 'oh waow i did x on a mac mini with 128gb ram'. That spec is beyond many, running on generally available resources (such as hardware one might have laying around their house) do not seem fit for the purpose, so its back to building a new machine (gl when ram is worth 2x its weight in gold), or buying a $1000+ mac mini, or other device. Any low end system cant turn out tokens fast enough, or doesnt have the resources for context or processing.
Local ai is not ready, and if you think it is, prove me wrong with a detailed guide running commodity hardware with complete setup steps that can use a decently sized model.
I spent 2 weeks trying to get anything running - 8gb RX550XT, 12gb ram, 8core cpu. I even tried turboquant to lower memory utilization and still couldnt even get a 3B or 4B model loaded, and anything lower wont suit my needs (3/4B are even pushing it).
"Local AI is not ready" > proceeds to run a 7 year old budget GPU
You're like the kid showing up to a test without a pencil.
It's ridiculous for you to suggest that an advanced AI model needs to run on your budget 7 year old graphics card that is already out of date for even today's gaming. My parents spent $2500 on a computer in 1995 and that was a 166Mhz Pentium 1. If they spent that money today it would be $5261. Think of what you can get for amount of money. Then you're over here trying to say a budget graphics card needs to somehow compete with the bleeding edge of computer innovation.
You do, in fact, need to spend money on appropriate gear if you expect to participate.
I've played with SD plenty. CPU even becomes manageable at low resolutions. But uh CPU/GPU is starting to blur now with these new AMD inference CPUs with built in GPUs. And ARM based machines like Macs. I wish more people on HN were using this stuff so we could have fun conversations about it instead of arguing over whether or not we should even be using these tools.
When Stallman was getting started writing emacs in the early 80s, Unix machines were vastly out of reach price wise for the common home user, but he did his open source work anyway, and eventually the 386 came along.
TBH I never understood people trying to run LLM locally. Just rent a powerful machine in the cloud for few hours. It's cheap enough, because you don't need to own a hardware. It doesn't introduce a dependency because there are hundreds of hosters. It doesn't compromise your data, because nobody would extract data from your VM, not until you're under an investigation, anyway, and even in that case just use different jurisdiction.
Spending humongous amount of money to get machine that'll felt obsolete in 2 years? I don't know.
Clearly havent seen what enterprise hardware is like these days... sure, the OS takes 5 seconds.. but the hardware can take 10 minutes in some cases now glares at hpe gen11 systems. Its seriously bad now. The amount of power and time backround hardware level tasks now take has significantly increased over the last 10 years. Even the ancient dell r710 i have sititng in a closet collecting dust boots faster than todays hp gen11's.
We waste a ton of energy on ineffeciencies in hardware and software today all because we managed to "just go faster".
There's some controversy about what would be better, universal basic income (UBI), or universal basic jobs[1]. But either one surely would be better than the status quo.
As automation via AI and the autonomous robot workforce slowly comes online, both of which are happening, these "basic jobs" will also become obsolete or automated. Jobs will increasingly shift to the STEM and away from workers on industrial lines or there will be extremely limited numbers of workers basically acting as overwatch; this can already be seen in a number of places where large factories are basically managed by 10-20 people unless theres large construction/expansion. This will become more prominent, as can be seen by the dark factories already in the works or partially operational. Many industrial jobs are going away, and people who have been in those industries their whole lives will not easily find other careers. UBI guarantees you can at least minimally survive without work, which is important during those periods where people are having to learn new industries to survive as their jobs are removed by automations.
Manufacturing isnt going anywhere, but it will increasingly take fewer and fewer people to operate and build many things.
UBJ would work in some cases where your knowledge could be (relatively) easily applied in other places, but we basically already have a partial implementation of UBJ, you can easily get employment assistance from almost any state in the US, but its also trying to do the same for 5+% of the human workforce at the same time. Sometimes it can be many weeks or even a few months to get something; what can a person already not making a lot do in that case? With UBI the stress is alleviated, and can help bolster the persons living situation when out of work.
Uggggggly car design, but the notion, i enjoy. Ive always been a fan of the retrofuturistic designs that people of the past had in mind for the future.
As such, i decided to start a new project a month or so ago.. Ive been working to build a model car kit from scratch, and i mean, from scratch. CAD to injection mould to kit and instructions. Im still in the cad steps, but its been pretty enjoyable so far. I am currently designing a retro coupe, which is powered by reactor. I might make it flying style, where the wheels are plasma thrusters instead of wheels.
uhg, good job but i regret clicking it! It made my eye twitch slightly, reminding me of a software i occasionally have to deal with called OpenLM.. its truly truly the worst webexperience ever had. Have you ever wanted a windows ui... but for a browser based application? No? Me either, and yet here this thing is!
Its like they took a look at the windows 98 ui and decided thats how their app should be, exactly. It even has a start button, and everything opens as a "window" in the browser screen.
And even worse, i initially thought it was some theme someone applied, but no, it was literally intentionally designed like that, and theres no alternative theme or formatting
Noob here. If you dont mind ive got some questions for you!
Ive recently started messing with the idea of making my own model car kits as a hobby. I understand a lot of the basics, but have never done anything like this before.
Im obviously not going to make kits in mass, but, i plan on doing injection molding using polystyrene. I do not currently have a cnc, but have been eyeing a SainSmart, though they say "can do metal under certain circumstances", but doesnt cover any of those circumstances. I also was looking at various injection machines and the price for entry is insane to me - $1000 for something that would probably burn your house down.
Anyway, to my questions..
1. Suggestions for a hobby cnc that can work aluminum? Id be willing to go as far as $2kUSD, unless theres something more that you think would serve me significantly better
2. Suggestions for a hobby injection machine that can do ~60-100g shots, that wont try to burn my house down, and doesnt cost a ton?
3. Any tips or thoughts for someone diving in to this?
4. Things i should purchase for QoL with cnc or injection molding?
5. Where does one buy materials (in hobby quantity) like aluminum block stock and polystyrene pellets?
Those are all things I've spend some time in, I'm not sure what to say. Learning curves for each one of those things are pretty dang steep. With AI you can probably speed up learning curves, but I think you will still go on many dead ends.
On the small CNC that will work with aluminum... There is a whole tradespace around how small of a feature you are trying to mill vs spindle speed vs machine stiffness & spindle runout. If you were to get something like a HASS you can sorta do it all, but when you get into the hobby stuff, you need to be very certain about what smaller set of machining limitations you will be dealing with and if they will still get you where you need to go. You need to work backwards from what actual tolerances you need to hold for the downstream thing to be able to work. (For instance, if you are making an aluminum mold, when you machine it, you will most likely be repositioning the work piece... if your machine isn't square enough so that when you flip the part on it's side or upside down, then do your next op, the part may not have been square to begin with, so now you have something that won't match the other thing you are trying to mate to.)
I build a 2'x5' 3 axis with ATC, starting from a CNCdepot concept and did my own control. I probably spend as much money on precision straight edges, levels, 90deg blocks, lapping tools, etc that were required to build a machine that could hold tolerances to 0.001", which is probably where you need to be landing to have molds that work.
I guess I am just trying to say it's a very big and ugly can of worms you are opening up.
Before buying anything, you might just want to try using firstcut/protolabs. They will machine the aluminum molds and mold the parts for you. Price per part is not going to be pretty, but it's going to be way less than spending thousands on machines that will never get you to where you want to go.
As for "desktop" molding, there is some startup now pushing their kludged together machine, maybe that is the one you are referencing. I'd stay far away from that thing. I think they were charging a couple $k for it, I feel like you need to be at 5x-10x that for anything reasonable. But at that point, the amount of power and infrastructure you need is well outside anything you'd want to put in your house or even garage. Don't really have a good answer for you here.
One thing to look at, if you are doing those model kits where like 20 parts are all in one flat sheet and you twist them to remove them, people are starting to make these on FDM 3D printers, which might be worth looking at. Now you can prototype and do production on the same machine, which at your stage, is the right place to be.
Sega was also the pioneer of online gaming, in a sense. SEGA channel was a service provided by your cable company. You got an adapter that went into your SEGA and had a coax that went to your tv box. Every month you get access to 20-40 games, all of which were a range of titles everyone could enjoy. There was no multiplayer, except locally. You could even plug in games you owned without removing the adapter. Also had the bonus of the ability to play the sonic version with the red sonic, by plugging in one of your sonic versions, when the adapter had the red sonic game expander available in its game list.
To me it still holds as one of the coolest technologies (aside from a computer of course) from my childhood. Managed through tv channel sideband data.
Ive been trying to run local, effectively followed this guide (before the guide existed), and have not had any success. Llama builds fine, and then when i start it up, it just indefinitely spins its progress bar. I left it sit for 3 days and nada.
Running on an 8core 12gb ram vm, which has an amd rx5500xt (8gb) passed through. ROCm built, llama built with the correct flags.
I assume this would break observability through existing methods, right? If you were to strace a process that has been patched, would you see regular syscall data (as if it wasnt patched) or would your syscall replacement appear along the way?
Good question. I didn't cover this in the post — the binary doesn't run on the host kernel directly. It runs inside a lightweight KVM-based VM with no operating system. The shim is the only thing handling syscalls inside the guest. So strace on the host wouldn't see anything — no syscalls reach the host kernel from the guest. From the host side, the only visible activity is the hypervisor process making syscalls on behalf of the guest.
Inside the guest, there's no kernel to attach strace to — the shim IS the syscall handler. But we do have full observability: every syscall that hits the shim is logged to a trace ring buffer with the syscall number, arguments, and TSC timestamp. It's more complete than strace in some ways — you see denied calls too, with the policy verdict, and there's no observer overhead because the logging is part of the dispatch path.
So existing tools don't work, but you get something arguably better: a complete, tamper-proof record of every syscall the process attempted, including the ones that were denied before they could execute.
I'll publish a follow-on tomorrow that details how we load and execute this rewritten binary and what the VMM architecture looks like.
Local ai is not ready, and if you think it is, prove me wrong with a detailed guide running commodity hardware with complete setup steps that can use a decently sized model.
I spent 2 weeks trying to get anything running - 8gb RX550XT, 12gb ram, 8core cpu. I even tried turboquant to lower memory utilization and still couldnt even get a 3B or 4B model loaded, and anything lower wont suit my needs (3/4B are even pushing it).
reply