This is a standard Lisp idea, but I think it's kind of off base.
Lisp has this idea of creating a language in which you can solve your problem. That's fine, but the Lisp people mean something rather different than the rest of us do.
When I "create a language to solve a problem" in, say, C++, I create some nouns (objects), some verbs (methods), maybe some adjectives (other objects or flags) and adverbs (more flags). Then I can write my application using that "language", but using normal C/C++ syntax.
When Lisp people say "create a language to solve a problem", they mean "write completely different syntax". The article gives some examples.
Why would I want to do that? Well, I might want to explore a new paradigm - some new thing like aspect oriented programming, say. I don't have to wait for a language that implements it, I can just do it myself. This is great for academic research projects, but not nearly so great for production code. (Bringing new people up to speed becomes a much longer process, if your code outlives the original developers.)
But I might have to create new language features just to get anything done in Lisp. One example is LOOP. It's a macro, because Lisp without macros doesn't have very good looping ability. "It's a building material" partly in this sense: It's not a very good tool. It's not all that usable until you add the parts that, in most other languages, you already get.
Now, does LOOP do more than a C-style for or while loop? I doubt it, but perhaps it does it more neatly. But C gives you 90% of what you need without a macro, whereas Lisp gives you 10% without a macro. If you need more in C, you can do it, even if it's a bit clumsy. But in Lisp without macros, it's horribly clumsy all the time.
(Yes, I know that there are Lisp and Haskell types who seem to regard writing a for loop over a container as a great waste of programmer time. I think that they are mistaken.)
You can do some amazing things with Lisp macros. Paul Graham gives the example of writing an extension language for Viaweb that Lisp macros turned into Lisp code, which was then run on the server. That's really slick (though in the current situation, you have to watch out for security issues unless you validate that file very carefully). But if he had chosen another approach, what would change? The macros have to turn the file syntax into Lisp syntax. If he had written an ordinary parser, he would have had to do the same. The only difference is that, by using macros, he let the Lisp compiler do the grunt work of the parsing.
TL;DR: Lisp needs macros to be usable. I'm not convinced that C does.
> When I "create a language to solve a problem" in, say, C++,
Most of the time you just add words to a language by adding verbs/adjectives/adverbs. It's not a new language.
> One example is LOOP. It's a macro, because Lisp without macros doesn't have very good looping ability.
That's backwards thinking.
LOOP is a macro, because macros are the way to implement code transformations in Lisp. Lisp has other iteration constructs, which are implemented as functions (MAP, REDUCE, ...).
Common Lisp is also its own compilation target. So at the very bottom there is only one iteration construct provided: GOTO. The rest are macros/functions on top of that. The language a compiler needs to understand is thus very small. The built-in extension mechanism then allows almost arbitrary code transformations. This is used by the language implementation AND the user. The user/developer has access to the same facility to implement code transformations in applications or libraries.
This is great for production, since you don't have to wait for some committee or a benevolent dictator to implement the language feature you need. Instead of waiting for a new iteration facility for months or years, you can implement it in an afternoon. Productivity goes up. You don't need to wait for tools generating code or more compact notations reducing boiler plate code - just implement it yourself. You also don't need external preprocessors - just use Lisp. You can also debug/extend your code transformation using the same development tools, instead of maintaining external preprocessors. It also enables complex programs to have comparatively small code bases. Which often makes maintenance easier.
TL;DR: Lisp gives the user more expressive power and trusts them.
In Lisp, you can write it in an afternoon. But in Lisp, because the language (sans macros) doesn't give you much, you have a bunch of things that you have to do that way. In C, because the base language gives you more, you have enough that you don't have to write the features that you need. (Granted, this doesn't work if your goal is to reduce boilerplate code to zero...)
> But I might have to create new language features just to get anything done in Lisp. One example is LOOP.
LOOP is a standard part of Common Lisp, so whatever it is it can't be an example of how Lisp-as-it-comes isn't good enough and you need macros to bring it up to standard.
> Now, does LOOP do more than a C-style for or while loop? I doubt it
In the sense in which a C-style for or while loop doesn't do more than goto, you're correct. Otherwise, of course you're incorrect; LOOP does a lot more than for and while do. (Random example: (loop for i from 1 to 10 collect (* i i)) makes a list of squares. No instance of C's for or while does anything like this.)
> Lisp needs macros to be usable. I'm not convinced that C does.
Could you give an example of something that C lets you do "out of the box", that Lisp (let's say specifically Common Lisp) doesn't, but that Lisp could be made to do using macros? I don't think I can think of any; there are things you can do better in C than in Lisp (e.g., close-to-the-metal bit-twiddling, or interfacing with other things written in C) but they don't have anything to do with macros.
LOOP is a macro, right? It's a standard macro, sure. But the point isn't standard vs. non-standard, the point is macro vs. non-macro. Without any macros, Lisp can't do loops very well. That's my point - Lisp without macros isn't very useful.
>Random example: (loop for i from 1 to 10 collect (* i i)) makes a list of squares. No instance of C's for or while does anything like this.
for (int i = 1; i <= 10; i++) list.push_back(i * i);
Yeah, it's C++ using STL rather than straight C. (I suppose you could argue that C lacks a list type and therefore isn't very useful... but I could create a linked-list struct and write a push_back function. For that matter, I could just have
int list[11];
for (int i = 0; i <= 10; i++) list[i] = i * i;
I don't really think that list-vs-array is really relevant to the relative power of the loops in C and Lisp.)
> Could you give an example of something that C lets you do "out of the box", that Lisp (let's say specifically Common Lisp) doesn't, but that Lisp could be made to do using macros?
Well, macros produce Lisp code, so there is nothing that you can write in a macro that you cannot write in straight Lisp. But in non-macro Lisp, loops are really painful to write - the LOOP macro is part of the standard for a reason.
> But the point isn't standard vs. non-standard, the point is macro vs. non-macro.
But why?
Imagine an alternate world in which all the things in the Common Lisp standard that are currently defined to be macros are special forms instead. I claim that (1) in that world, the criticism you're making would not apply, and (2) that world's Common Lisp is (a) not better in any way than our world's and (b) actually slightly worse, because code-walking macros would be more painful to write. Doesn't that indicate that there's something wrong with your criticism?
What I'm not seeing is how the fact that some standard features of Common Lisp are defined to be implemented via macros is a weakness, which you seem to be arguing it is.
On LOOP: Sure, you can use a C for loop to build a list of squares (but, note, it doesn't in fact do the same thing as the Lisp code and couldn't possibly, because (1) the Lisp code is an expression and a for loop is a statement but not an expression, and (2) that invocation of LOOP makes a new list and returns its final value rather than appending to some specific list that's already been created as your kinda-sorta-parallel code does). But that's what I meant about goto. You can use goto to do what a C for loop does. If you wouldn't say "Does for do more than goto? I doubt it" then you shouldn't say "Does LOOP do more than for? I doubt it".
> But in non-macro Lisp, loops are really painful to write
But "non-macro Lisp", if by that you mean something like "Common Lisp, with all the features defined to be implemented as macros taken out", is a thing that doesn't exist and that never would exist. (If for some reason you were implementing a Lisp without macros, and if you actually intended it to be a good general-purpose language, then of course you would include some decent looping facilities.)
So this is not an example of something that Lisp doesn't let you do "out of the box"; LOOP is right there in the box. As you say, LOOP is part of the standard for a reason.
Well, my initial claim was that Lisp needs macros in a way that C/C++ doesn't. You are now stating that Lisp could have been written in a way that it didn't need macros to be usable. And you're right, it could have been. But I don't think that that's the point.
The point is not that implementing things as macros is a weakness. It's that the language (as written) needs those macros. And C, as written, does not.
But I suppose from your point of view, my argument is a distinction without a difference...
I understand what you are saying. Its more of a claim that C/C++ does not need this project, as macros are not a necessity of the language.
I think the one thing that this project could provide (though due to the lack of function call access inside of macros that IS provided in Lisp macros maybe not) is to take that 10% of functionality that C/C++ does not provide and make it much more easily accessible. Of course this is always the difficult situation to discuss when first learning about Lisp macros because so much "standard" functionality already exists but I assume the old trusty could be applied here.
In many tutorials for learning Lisp macros the idea of "Lets say you wanted a statement (when <test> <expression>)." Its something that doesn't exist and the developer can add it in during development. You do not have to wait for someone else to implement this into your compiler.
This is a case where Lisp _needs_ macros to implement and where C/C++...can't do anything. Yes you can make a function that is similar to the format required but that is kind of the point. Its a hack to get the functionality and not something that looks natural.
I agree, there's that 10% in C/C++. And, you're stuck. But working around that 10% is an annoyance, not crippling. (In over 20 years as a professional C/C++ developer, I have never hit something that I couldn't write. "when <test> <expression>", for example. Yes, I can't write that exact syntax, but I don't care. I can get the same result, though perhaps slightly more clumsily.)
But Lisp without macros is crippled.
But, as lispm and gjm11 pointed out, Lisp could have been implemented with those macros as special forms instead, so Lisp wouldn't have to be crippled without macros.
My main point: I think the Lisp crowd overestimates how limiting it is for other languages not to have macros.
Well, yes, it is. You can turn Common Lisp (as it is) into a "language that doesn't need those macros" simply by going through the spec and changing everything that says, e.g., "macro DEFUN" to "special form DEFUN". The result would be a language that's no more powerful, no easier to implement, and marginally less convenient to use.
So how can "needing those macros" be a weakness, if the minimal change that makes the language not "need those weakness" improves nothing and actually makes the language a little worse?
Lisp has this idea of creating a language in which you can solve your problem. That's fine, but the Lisp people mean something rather different than the rest of us do.
When I "create a language to solve a problem" in, say, C++, I create some nouns (objects), some verbs (methods), maybe some adjectives (other objects or flags) and adverbs (more flags). Then I can write my application using that "language", but using normal C/C++ syntax.
When Lisp people say "create a language to solve a problem", they mean "write completely different syntax". The article gives some examples.
Why would I want to do that? Well, I might want to explore a new paradigm - some new thing like aspect oriented programming, say. I don't have to wait for a language that implements it, I can just do it myself. This is great for academic research projects, but not nearly so great for production code. (Bringing new people up to speed becomes a much longer process, if your code outlives the original developers.)
But I might have to create new language features just to get anything done in Lisp. One example is LOOP. It's a macro, because Lisp without macros doesn't have very good looping ability. "It's a building material" partly in this sense: It's not a very good tool. It's not all that usable until you add the parts that, in most other languages, you already get.
Now, does LOOP do more than a C-style for or while loop? I doubt it, but perhaps it does it more neatly. But C gives you 90% of what you need without a macro, whereas Lisp gives you 10% without a macro. If you need more in C, you can do it, even if it's a bit clumsy. But in Lisp without macros, it's horribly clumsy all the time.
(Yes, I know that there are Lisp and Haskell types who seem to regard writing a for loop over a container as a great waste of programmer time. I think that they are mistaken.)
You can do some amazing things with Lisp macros. Paul Graham gives the example of writing an extension language for Viaweb that Lisp macros turned into Lisp code, which was then run on the server. That's really slick (though in the current situation, you have to watch out for security issues unless you validate that file very carefully). But if he had chosen another approach, what would change? The macros have to turn the file syntax into Lisp syntax. If he had written an ordinary parser, he would have had to do the same. The only difference is that, by using macros, he let the Lisp compiler do the grunt work of the parsing.
TL;DR: Lisp needs macros to be usable. I'm not convinced that C does.