To be fair there are a lot of possible systems from which you could derive infinite complexity. There are an infinite number of true statements that can be discovered after all. Most are not particularly interesting or useful. You could come up with proofs for more and more specific behaviors that occur in turing machines under specific conditions. Or cellular automata. Or any number of arbitrary systems. But most wouldn't be relevant to the real world. The cool thing about numbers is that there are a lot of possible applications to the real world. But there are also an infinite number of trails you could follow with mathematics that lead nowhere useful at all.

Automated Mathematician tried to explore mathematics and find interesting concepts the same way mathematicians do and using similar heuristics. And it got fairly far, discovering numbers and primes and Goldbach's conjecture. But eventually it just started discovering useless concepts which had no apparent relevance to anything and were pointlessly complex or specific.