Bitgrid[1][2] is a continuation of an idea I've been carrying around since the 1980s, and tweaking from time to time. If I can maintain focus long enough, I intend to learn RTL, VHDL or Verilog well enough to get it entered in one of the Google open source chip shuttles. I think this approach to computing if applied to neural networks may offer a cheap way to get far more compute, perhaps exaflop performance out of a single chip.
It's an FPGA without routing hardware, which on the face of it, is the stupidest thing to do. However, because of the clocked nature of the bitgrid, you all data only has to travel to neighboring cells, so lines will be short, and clock rates should be able to be up in the gigahertz range. Instead of worrying about how quickly you can get a signal from one side of the chip to the other, the latency will be 1/2 of the number of cells across, if the signal takes a straight line.
However... everything along that path can be compute, and routing should be trivial. It avoids the trap with GPUs where they are Turing complete, thus hard to reason about. Like an excel spreadsheet, you can track dependencies, and know exactly where a given bit came from. The chip as a whole, on the other hand, is Turing complete.
If anyone knows how much energy in FemtoJoules an 4 bit latch and a 4 bit input LUT take up, and a static ram cell... that would help in estimating the real world power consumption/feasibility of this thing. I can't find a good answer anywhere.
Have you contrasted this with cellular automata? It seems like an advanced form of that type of construction. Could possibly benefit from some formalization.
Cellular automata apply the same rule across a grid, a bitgrid is an array of lookup tables, essentially stuffed between the cells of a static RAM. The RAM holds the values in the tables, and an latches maintain the state of each cell's inputs. Thus each cell in a bitgrid is individually programmable.
A provision needs to be added to be able to read and or write (override) inputs for debugging or other purposes, such as testing, updating constants, etc.
I agree that formalization is required. I've built an emulator, and hand coded some logic into it as a test, and can simulate a 1024x1024 grid at about 35 Hz on my desktop pc (half that on my laptop).
My near term goals are to be able to take an expression, run it through a set of tools to be written, and then feed it into the simulator, and run it.
I can guess at the energy required to change states, and thus get a rough estimate as to power usage/efficiency/speed, etc. If I get myself to the point where I've got a chip designed, I'm sure the EDA tools can give me far more accurate numbers.
Oh, I meant formalization as in, this is clearly not either a turing machine or a lambda calculus, but as it appears turing-complete, it must be equivalent, and could benefit from formalization in that computer-science/mathematical sense. Then we outsiders would be able to see what we're looking at.
You could consider a BitGrid to be a giant state machine, given any current set of inputs, and the program loaded into it, it is possible to know exactly what the next state will be. The connectivity turns it into a complex system. The number of possible states is somewhere between 1 and 16^cellcount
I'm not sure that it's possible to compute (in a reasonable amount of time, or maybe ever) the exact number of possible states a bitgrid may have, given its current program.
It's my conjecture that finding this exact number is equivalent to solving the halting problem.
---
Also, any computation in a Bitgrid is an acyclic directed graph. Because there is a delay between cells, you can only effect future states.
There are many ways you could do backprop... if you want to probe the values straight from the middle of the grid, you could do that. You could poke new values right into the middle of things (by using configuration/random access mode), but everything else is going to pause while doing so. Or you could ripple in new weights along with data.
To me, this feels like starting from where Turing stood and inventing a new type of computer. The sky is the limit.
It's an FPGA without routing hardware, which on the face of it, is the stupidest thing to do. However, because of the clocked nature of the bitgrid, you all data only has to travel to neighboring cells, so lines will be short, and clock rates should be able to be up in the gigahertz range. Instead of worrying about how quickly you can get a signal from one side of the chip to the other, the latency will be 1/2 of the number of cells across, if the signal takes a straight line.
However... everything along that path can be compute, and routing should be trivial. It avoids the trap with GPUs where they are Turing complete, thus hard to reason about. Like an excel spreadsheet, you can track dependencies, and know exactly where a given bit came from. The chip as a whole, on the other hand, is Turing complete.
If anyone knows how much energy in FemtoJoules an 4 bit latch and a 4 bit input LUT take up, and a static ram cell... that would help in estimating the real world power consumption/feasibility of this thing. I can't find a good answer anywhere.
[1] https://github.com/mikewarot/Bitgrid
[2] https://bitgrid.blogspot.com/2005/03/bitgrid-story.html