Answers from 2017 Common Lisp experts

Answers from 2017 Common Lisp experts

This post is a compilation of answers to the questions posted in a blog post titled Questions for 2017 Common Lisp experts, posted by anticrisis.

I have decided that the questions asked in that post are good enough to deserve a response post that gathers the collective knowledge of the Lisp community in a FAQ of sorts.

It's somewhat baffling to say that "experts", whoever they are, have gathered to respond to this question - I simply assume that the people who have voiced their opinion on the topic - regardless of who they are - know what they are talking of, and therefore their comments should be, and have been, included here.

This post includes answers shamelessly scavenged from the reddit thread discussing these comments along with answers that I was able to collect from other places and people - to be precise #lisp @ Freenode and the Lisp Discord server.

The responses were slightly edited where necessary to add context, punctuation and capitals; I am to blame for all unintended changes to meaning that it might have caused. On the other hand, even if the answers were mostly duplicating each other, I have decided to keep them all; I do not want to editorialize here, but instead try to (perhaps a bit experimentally) show the inner workings of the Lisp community's hivemind and outline the questions that are perhaps the most controversial and eliciting mixed views and responses among the community. (Why? Because I can.)

This post also includes my own answers mixed in between all others.


Table of Contents

This is a copy of the original blog post, which also happens to be a table of contents for the below document.

Libraries

  • Why don’t libraries have version numbers?
  • Why is adding libraries to Quicklisp a manual process that requires human review? Is there an alternative non-curated registry?
  • Why aren’t ASDF dependencies automatically fetched if missing?
  • Why do so few libraries use GitHub?

Open source

  • Are there serious open source efforts to modernize documentation?
  • Are there serious open source efforts to modernize libraries?
  • Are there efforts to encourage open source contributions?
  • My apologies, but I’d never even heard of GitLab prior to seeing all these libraries hosted on common-lisp.net. Why not GitHub?

Compilers

  • Why hasn’t the open source community standardized on a single implementation, i.e. SBCL?
  • Would there be any benefit to extending SBCL beyond the Common Lisp standard to better support modern language features and paradigms?

Functional programming

  • Why aren’t there more libraries using a functional, immutable paradigm? Most seem to use many dynamic variables and large functions spanning multiple screens, with variables mutating throughout
  • Are there any functional programming advocates in the Common Lisp open source community?

Innovation

  • Are there any efforts to bring innovative libraries from the Clojure (or elsewhere) community into Common Lisp?
  • Is there software design innovation occurring in the Common Lisp community or is the community primarily focused on maintenance and tuning?
  • Is it worth the time and energy to develop new libraries to modernize Common Lisp – or perhaps a single extended implementation of it, like SBCL – or would that time and energy be more productively spent in a non-Common Lisp community?

On-boarding

  • Why is on-boarding new developers so difficult? For instance, why isn’t Roswell linked to from every About Common Lisp page?
  • Why hasn’t the Common Lisp Foundation updated the common-lisp.net home page since 2015?

Community

  • Why are there so few instructional or advocacy bloggers?
  • Why can’t Google search find many good resources? I.e. a search for “Common Lisp” is not that helpful.

Libraries


Why don’t libraries have version numbers?

defaultxr

They do. Typically declared in the .asd file. It's up to the library's author to specify/update those version numbers of course.

Xach

Everything uses the latest of everything.

Quicklisp makes it pretty easy to use libraries of the same vintage that are known to work together.

qlot also makes it pretty easy to use a variety of upstream sources and stick to a known-working set of libraries, and reproduce that set on other machines. (I haven't personally used qlot, but it seems like a sensible idea.)

ruricolist

Quicklisp provides more assurance than you might expect. The criterion for inclusion, and maintenance, of a project in Quicklisp is that the project successfully load in SBCL alongside all the libraries in that Quicklisp distribution, and load without any errors or warnings. SBCL has extensive type inference and can catch and warn about many potential issues at compile time. And because of the pervasive use of macros in CL, successfully loading a library usually exercises a lot of code paths in its dependencies. To a surprising extent, "if it loads, it runs."

Qlot isn't something a library would use. You use it to set up the dependencies for an application. My approach (with TBRSS) is this. Obviously, every time I upgrade to a new Quicklisp dist, I run a test suite to make sure everything is working together. On the rare occasion there's a problem, I either pin the offending library to an older version (with Qlot) or I fork it and fix it, again using Qlot to pull from the fork until the fix makes it into the next Quicklisp dist. And of course I also use Qlot for dependencies that are not, for whatever reason, available in Quicklisp.

Aidenn0

They do. In fact all ASDF systems have a version number. Just very few people use them.

phoe

The question is both incorrect, in a way, and correct, in a way.

The incorrect fact: as we can see in a ASDF Manual - ASD systems each have the capability to declare a version number, and multiple ASD systems use this facility.

The correct intuition: some systems simply do not use it, with the most notable example being the most popular Quicklisp library - Alexandria. Additionally, many systems simply do not use versioned dependencies and instead opt for using the latest released version of a given library.

This might look very different compared to public repositories like NPM. I will list the main differences that I see, which make this kind of approach viable:

  • The JS/NPM community is much larger than the CL/Quicklisp community,
  • The code volume in NPM is hundreds, if not thousands of times larger than in Quicklisp,
  • The libraries are changed and modified much quicker and more often in the JS world compared to the CL world.

This way, there is not much space where breakage may occur the moment a change in a dependency breaks projects that depend on it. Though, I'm not the most proper person to talk about this; Zach Beane runs monthly tests on whole Quicklisp whenever he updates the Quicklisp dists, and because of that, he has most likely experienced much more in that topic than I have.


Why is adding libraries to Quicklisp a manual process that requires human review?

defaultxr

Quicklisp is mostly the work of one person, and it's still considered to be in beta from what I understand. I don't think Quicklisp is "curated" in a strict sense but it does require the libraries to actually compile and not print any warnings during compilation.

Aidenn0

Because there are few enough libraries that there hasn't been a scaling problem yet.

phalp

I think it's very good that Quicklisp is curated. Without curation, you get frivolous libraries like left-pad, and you get typo-squatting malicious packages like we heard about in npm last week. You can't assume a package is reliable just because it's in Quicklisp, but at least you can have some faith in the package list itself.

phoe

Because it was designed that way by the person responsible for all of Quicklisp - Zach Beane. The current process of adding a Quicklisp project into the repositories is a rather quick and painless one - there is no actual review of code and projects other than the two sole minimal assumptions that the project builds on SBCL x64 on Linux and has the ASDF author/name/license data correctly filled in.

Hexstream

The comment above is mostly right, but SBCL-only (or single-implementation-only) libraries are frowned upon, at least. Also, I would say "builds without errors or warnings".


Is there an alternative non-curated registry?

Aidenn0

Anybody can create a quicklisp "distribution" (which is a registry), though I'm not currently aware of any public ones.

phoe

In theory: everyone can create their own Quicklisp dist, perhaps with a backend that allows automation.

In practice: nobody I know has done that yet because of lack of demand for it. Quicklisp is simply good enough.

Hexstream

I've come across at least 2 custom Quicklisp dists recently, one was the cl21 one, I don't remember what was the other one... But yeah, extremely rare. They're partial dists with a few projects, though, not full-fledged quicklisp alternative dists.


Why aren’t ASDF dependencies automatically fetched if missing?

defaultxr

That's not what ASDF is for. ASDF is like make, Quicklisp is like your OS's package manager. Different tools for different tasks... However in the past ASDF did have functionality to download libraries that were missing (I believe it was called ASDF-Install?) but nowadays that is deprecated in favor of Quicklisp which is more complete and robust, from what I understand.

Aidenn0

They are if you are using Quicklisp. That being said, ASDF is solving a narrower problem of managing already installed dependencies. Quicklisp solves the problem of automatic fetching (which requires some sort of registry of packages).

phoe

ASDF is a local system manager and a build tool, not unlike make. Quicklisp is an online repository that automatically fetches dependencies and tells ASDF where to find them for compilation.


Why do so few libraries use GitHub? / My apologies, but I’d never even heard of GitLab prior to seeing all these libraries hosted on common-lisp.net. Why not GitHub?

defaultxr

Common Lisp has been around a long time, and many older libraries predate GitHub.

arvid

Most of them do use github. 1251 out of 1582 or 79% quicklisp projects use github. See the list here.

Aidenn0

Because so many libraries are older than GitHub.

GitLab is at least somewhat well known. The really short answer is that GitLab is open source. Some projects that were hosted on common-lisp.net prior to github existing have moved to github. Others prefer to be hosted on common-lisp.net; for most of these projects SourceForge, not GitHub was the alternative to common-lisp.net when this decision was made.

kazkylheku

There are good reasons for open source project not to use GitHub.

GitHub is a company which operates a closed-source "walled garden" website with git hosting (which anyone can do with a host running GNU/Linux or BSD) plus various additional proprietary services.

If GitHub disappears, all you have is your git repos, because they are cloned all over the place; what about all the other GitHub specific data that you and your users put effort into?

Please drop the obnoxious mentality of questioning people who don't use GitHub.

It's not some axiom of the universe that "thou shalt use GitHub".

(Or even git, for that matter. Git is not the best version control tool in all requirements. One excellent technical reason for someone not to use Github is that they don't use Git.)

alcahd

Github's software is notably non-free. I find this contradictory and prefer not to use it where possible. Gitlab also has a few nice features not seen on Github such as the ability to run your own instance on a server. However, the lack of libraries on Github is mostly a symptom of age, rather than any conscious choice.

dzecniv

About github, to me the problem is to not have open registrations. Common-lisp.net uses their own gitlab, why not, but the registration is not open (one must ask) and so it drastically slows down or prevents participation.

The pb to me is using common-lisp.net where registrations are not open. This is a big pb that slows drastically participation down.

pjb

One problem is that github is github.COM and is located in the USA; gitlab on the other hand was gitorious.ORG and was located in Europe. At a time when network or github failures meant that European workers were stuck half a day without being able to push to the central repository, and considering that most CL libraries are freedom software, and not privative COMmercial software, it would seem more logical to gitorious.ORG than github.COM.

On the other hand, nowadays, gitorious has been bought by gitlab.COM, but there's framagit.ORG which is located in France.

There are also licensing restrictions that were imposed by github with some self generated FUD. (It seems they're cleared it up, but still, the point remains that if you use only one such repository, you're at the mercy of the US enterprise and their shareholder).

phoe

Why should they? As long as they are hosted in a place where they are fetchable, then they are reachable and downloadable by Quicklisp. Additionally, as long as they are hosted on some tool that allows GUI cooperation, such as Gitlab, Gogs or Gitea, then it is possible to create and submit pull requests to them in a convenient way.

It might not be fully related here, but the RedMonk graph says we're above the curve when it comes to GitHub repos.


Open source


Are there serious open source efforts to modernize documentation?

defaultxr

Yes. You may be interested in the Common Lisp UltraSpec project.

Aidenn0

Yes, the CLHS is not available under a permissive license, but the original specification (technically final draft specification) is. One effort I'm aware of is here.

phoe

Yes - the effort started by me is Common Lisp UltraSpec that has been rather stagnant for the last few months as I've been busy with other projects and so-called real life. But, the projet is somewhat lively despite its slowness.

Hexstream

There's a modern CLOS MOP rendition project underway. Unfortunately this thread is the best thing to link to about this right now, but I think it's very relevant to the question.


Are there serious open source efforts to modernize libraries?

defaultxr

There are plenty of people working on new libraries for Common Lisp. There are also projects to update older libraries. For example, there was recently a big fundraiser to update McCLIM, a native Lisp GUI toolkit. The fundraiser was successful and there have been regular updates posted by the leader of the project about the progress being made.

lispm

You might want to clarify this a bit.

As a possible partial answer to your question: CL code has a tremendously long 'shelf life' when compared to "modern" languages. Because of the standard, code written tens of years ago or more will, more often than not, work just fine, assuming it was written for a lisp that conforms to the standard. External dependencies of course change, so ymmv, but the code will be no less correct because of its age.

alcahd

Change happens slowly in the Lisp world, but at least part of that is because it's much older than other language families and many of those software design practices were already tried decades ago. Sticking with Lisp helps inoculate you against some of the more illusionary change in other languages, i.e. parading features that were already well known in the 80s.

phoe

Why modernize them? Are they broken or outdated? If they need maintenance, why not call it "maintaining them" instead of "modernization"?

A part of why Common Lisp as a language is beautiful is that portable Common Lisp code written 20 years ago is expected to work in exactly the same way in contemporary Common Lisp implementations.

Hexstream

In a way, any libraries that are in Quicklisp can be said to have a certain level of "modernity", as at least they're guaranteed to use ASDF, the current de-facto standard build system. They've at least been passingly looked at and built by someone in the last ~7 years, which given the relatively slow-moving Common Lisp ecosystem, is in some ways not that long ago. Although not all Quicklisp libraries may be actually suitable for use, for most people it can generally be assumed that any libraries that have been around for a while and are not in Quicklisp probably need some updates and can probably be ignored.


Are there efforts to encourage open source contributions?

phoe

One specific thing I know: some projects, such as McCLIM, have set up bounties for solving identified problems and lacks within their codebase.

Aidenn0

common-lisp.net exists largely for this purpose.


Compilers


Why hasn’t the open source community standardized on a single implementation, i.e. SBCL?

Baggers

In short, because people have different needs. Folks who want to embed in C applications will benefit from ECL, people who work in a Java stack can benefit from ABCL people, etc..

Aidenn0

The same reason that the Open Source community standardized on a single Linux distribution (or even Unix kernel BSD, Illumos, Linux &c.)

alcahd

The community has mostly settled on SBCL, many projects (Stumwm, for example), have dropped support for less used implementations. And for good reason, SBCL is significantly faster than anything else. However, other implementations have their special use cases and can target non-x86 architectures.

Hexstream

It largely has. SBCL is by far the most popular and best-supported Common Lisp implementation right now, and most newbies should use it by default unless they have very specific needs.

Xach

I used to think [that the community has mostly settled on SBCL], but the more experience I get, the less I think it is true. Every active CL has a number of users that cannot or would not switch to SBCL. They'd be more likely to stop using CL.

KDallas_Multipass

Me too. At the moment one thing I got used to with CCL was not having to know the size my heap would grow to before I started the image. I didn't even know that was a problem until I started trying sbcl with code that by design would consume a bunch of memory. I remember noticing that landing in the debugger in sbcl seemed nicer and richer than it would be when I was in ccl, and I'm aware of anecdotes about how sbcl might be a slower compiler for compiling faster code, and that it provides a lot of feedback that you can use to guide performance improvements. I appreciate that there are multiple mature implementations. I worry about the messages about how the windows implementation of sbcl is considered "unstable". I know if I had found sbcl first I would probably be saying different things.

phoe

Because it's much better and more varied this way. You can use one Common Lisp code and use it to reap whatever implementation's strengths you need to leverage at the moment. (Warning: this list of individual implementations' strengths is not exhaustive and is meant to be illustrative.)

  • SBCL has a compiler producing very fast code, capable of e.g. using SSE CPU instructions and doing atomic compare-and-swaps.
  • CCL has a fast compiler, as in, compilation is very quick there. It also has a native Cocoa bridge for OS X and has some nice debugging capabilities.
  • ECL compiles from plain C and runs anywhere, including Android.
  • CLISP also compiles from plain C and is an interpreter (optionally compiling to bytecode) that has very good native Unix interoperability and no compilation overhead, making it viable for scripting.
  • Clasp interoperates between C++ and Common Lisp code, allowing you to marry these two completely different languages.
  • Mezzano is a FREAKING OPERATING SYSTEM WRITTEN IN COMMON LISP THAT RUNS IRC, QUAKE AND DOOM.
  • ABCL runs on the JVM, meaning that it runs everywhere Java does.
  • mocl runs on iOS/Android and allows you to develop for mobile platforms.
  • LispWorks allows you to write very portable GUI using its CAPI.
  • ACL has a concurrent garbage collector, which means very small GC pauses.

Would there be any benefit to extending SBCL beyond the Common Lisp standard to better support modern language features and paradigms?

defaultxr

SBCL and most other CL implementations do include extensions. For example, threading is not described in the standard, but SBCL provides an extension to allow Lisp programs to use threads.

Baggers

As you've seen above, most implementations do extend. But it feels like the 'modern language features and paradigms' part of the question is suggesting updating the standard. One the beuties of CL (and some other members of the lisp family) is the fact you dont need to force the entire community to do something to get change. You can see projects like CL21 as an example, once again choice is key and the non-dictating nature of CL make it a good home for people with a wide range of priorities.

You can see things like grey-streams, cffi & bordeux-threads for examples of things that have become defacto standards in the community and also to projects like optima and named-readtables which provide feature to CL as libraries which in other languages would require a new version of the language.

It's also wonderful that, if you make a better way of working, you can release it as a library and let people decide if you were right :) and if they dont like it you lose nothing. That and that standards compliant code from 30 years back still works just fine as we've been gaining all these nice features.

There are some things that can't happen without buy in of every implementation. Here are a few examples.

Aidenn0

Some of this exists already. The lisp community is very diverse, so attempts to modernize the library are hard to do. There have been many attempts; see one recent discussion.

alcahd

As the for the CL standard, basically every implementation already extends it, which is part of the reason there were portability issues in the past. The standard is another product of age and omits a number of things you'd expect from a modern language.

PuercoPop

Lisp is all about extensibiliy. Case in Point: CLOS. One can extend the object system in terms of the same object system. (Modularity not so much imho).

phoe

You are confusing two things.

First thing: You do not need to extend the implementation to better support (most) modern language features and paradigms. Common Lisp already allows you to do so on any implementation.

Lisp does not have RegEx. So what do you do? You implement it in Lisp! Lisp does not have contracts. So what do you do? You implement it in Lisp! Lisp does not have pattern matching. So what do you do? You implement it in Lisp! Lisp does not have asynchronous programming capabilities like promises etc. So what do you do? You implement it in Lisp! Hell, early Lisps did not have CLOS. So what do you do? You implement it in Lisp!

Second thing: the implementations were and are being extended to support hardware and OS features that were unavailable previously. The most important examples I can think of are threads, sockets, OS interoperability and SSE CPU instructions for numeric/array crunching.

fiddlerwoaroof

as far as the "extend the standard" question goes, my sense is that the Python community has discovered that the standard library is where libraries go to die. Once a library gets incorporated into a "standard", it can no longer develop at its own pace, but only at the pace of the various implementations of the standard.

Whereas, if you implement the library in terms of other language features and maintain it, it can evolve as necessary. See http://www.leancrew.com/all-this/2012/04/where-modules-go-to-die/

Supposedly Rust people have also found this. https://news.ycombinator.com/item?id=12177545

dzecniv

did you see http://cl21.org/ ? It's a very important effort to me. It brings more functional and generic stuff, and fixes many CL oddities. It's very usable right now. But as always doc is scarce or missing. This is a problem I'm not sure the community is well aware of.

So you could have a look at https://lispcookbook.github.io/cl-cookbook/cl21.html for cl21 details. And it's a community project hosted on github, so… ;)

lispm

I personally would not use cl21.org , since it has some of the worst design of any CL library I have seen. It's ugly and misguided. The code is bad and poorly designed. One takes a huge performance and functionality hit. One can look at each function and discuss its design. Something which actually was done during the CL design, which was based on knowledge of prior dialects. You can find that the designers of Common Lisp both integrated knowledge from other dialects (Maclisp, Interlisp, Standard Lisp, ZetaLisp, ...) and tried to design a language with purpose.

The thing was written when the author did understand only 20% of Common Lisp, which is not a good base for a language redesign. But this is a common pattern, people not knowing too much of the language and with exposure to languages like Perl, Ruby and others, first want to redesign Common Lisp. Often they do it in ways which directly violates the spirit of the language and its built-in ways to change and extend the language. The only worse thing are people who actually understand Common Lisp and use it to create personalized language constructs, where each construct got a lot of thought, but them sum of all the thoughts isn't elegant and not fit for wider use. I'm not naming people, but some can probably guess who I mean. ;-)

It only supports just four implementations, though claiming to be written in portable CL...

As its page says it's 'experimental'. Experiments are fine, but it's an experiment which needs a garbage collector. Throw it away and start new. (Side note: this is not limited to Common Lisp.)

Imagine someone with exposure to Wing-Tsun and Taekwondo is trying to learn traditional Karate. Now after a year he is starting to design his own version of Karate with more effective moves, reusing tricks from Wing-Tsun and Taekwondo. But how big is the chance that this person will have understood the spirit and foundations of Karate? Alternatively one could study Karate until one has mastered it and having a Dan grade (a black belt) and then being able to redesign the moves based on actual background knowledge and first-hand experience.

The speed hit [of cl21] is acceptable for a Ruby user, because Ruby is about 103 times slower than a typical Common Lisp natively-compiled implementation and thus cl21 becomes as slow as Ruby. Works fine as a scripting language though.

The author does not understand how actual implementations extend the language in ways that makes sense and what consequences this has for certain language constructs.

Redesigning (Common) Lisp is a popular sport and some invested a lot of time: Dylan, ISLisp, Eulisp, Arc, Clojure, CLtL3, CL21, ...

The author wrote a lot of software in a very short time, but he didn't took the time to learn the language on a deeper level, which would be required for a language (re)design. That works for him and some others, but please don't assume that everyone likes the design, the implementation or even its goals. The CL21 implementation and design has never been discussed on a wider level.

Promoting an experimental design as the base for general purpose libraries is possible, but don't be surprised when others point out its flaws. I'm also not a fan of the gradiose claim of a CL in the 21st century.

I couldn't even be motivated to port it to other implementations, because I would not want to use it.


Functional programming


Why aren’t there more libraries using a functional, immutable paradigm? Most seem to use many dynamic variables and large functions spanning multiple screens, with variables mutating throughout

defaultxr

CL is a multi-paradigm programming language. It existed long before purely functional programming a la Haskell became the hip and cool thing to do. CL is not about forcing a specific paradigm on you. If you want that, no one is stopping you from using Clojure, or writing your own CL libraries in a more purely functional style.

Aidenn0

Because the fraction of people writing pure immutable code in Common Lisp isn't much higher than for any other general purpose language, so the absolute number of people to work on such libraries is much smaller.

alcahd

Common Lisp has always been a multi-paradigm language, there are other Lisps such as Scheme if you want to go down the FP road. It's fair to say that the syntax errs towards a functional style, but the language itself is flexible enough to accommodate whatever the user thinks is best. The OO system, CLOS, is worth looking into, it's very powerful and somewhat unique, using neither message passing nor the imperative systems seen in Java/C++.

phoe

Has functional programming been found ultimately better than anything else, or is it just trendy nowadays?

Common Lisp allows the programmer to mutate if they consider it to be proper. Common Lisp also allows the programmer not to mutate if they consider it proper. This language does not enforce a particular paradigm on you. If you want it to enforce a particular style on you, nothing prevents you from creating such limitations yourself.

Common Lisp is a multi-paradigm programming language that does not limit the programmer to using only paradigm X or Y. You can write imperative code, you can write object-oriented code, you can write functional code, you can write declarative code, you can write actor-based code, you can write contract-based code, hell, you can go back to 1964 and write (TAGBODY 10 (PRINT "HELLO WORLD") 20 (SLEEP 1) 30 (GO 10)) because linear programming is supported as well and all iterating constructs I know actually compile into GOTO-like structures such as that.


Are there any functional programming advocates in the Common Lisp open source community?

phoe

Betcha there are, but as I said before, Common Lisp is not a functional language. The style commonly advocated is "mostly functional" and tends to be a mix of imperative functionalities like LOOP, the Common Lisp Object System (CLOS), and functional capabilities like higher-order functions.


Innovation


Are there any efforts to bring innovative libraries from the Clojure (or elsewhere) community into Common Lisp?

Aidenn0

There are many places where people have seen a library in another language and brought some of it over to Common Lisp. The community is much smaller than other language communities, so there is limited programmer time for working on these.

dzecniv

There are. What are you thinking of I saw cl-arrows for threading macros, fset for functional data structures (preceding Clojure though); what'd you like?

phoe

Let me ask an apparently more crazy question: are there any efforts to bring innovative language features from the Lisp community into Clojure (or elsewhere)? Like, the conditional (IF/THEN/ELSE) construct? Higher-order functions? Dynamic memory management? Garbage collection? Homoiconicity? ``

A few of the examples are Clojure's arrow macros, transactional memory, monadic parsers, single interface for parallel execution of functions.


Is there software design innovation occurring in the Common Lisp community or is the community primarily focused on maintenance and tuning?

Aidenn0

See vslevod's talk I linked to earlier and find out if rutils is moving in the way you like.

phoe

It might not be obvious, but Lisp is constantly sprouting new leaves and branches. Two huge projects that I know that regard Lisp design innovation:

  • Clasp is a young implementation that compiles Common Lisp and C++ into one code for the sake of science, chemistry and general usage.
  • SICL is a growing implementation that effectively reimplements Common Lisp in a modular way and a completely new code style.

Overall, the community is alive. New libraries are being created and uploaded, new projects are being written in Lisp, new bindings to foreign libraries are established.


Is it worth the time and energy to develop new libraries to modernize Common Lisp – or perhaps a single extended implementation of it, like SBCL – or would that time and energy be more productively spent in a non-Common Lisp community?

defaultxr

This largely depends on what you need from a language. CL provides many features that other languages still lack, but there are of course areas where other languages are better suited for a specific task. Sometimes the gap is larger or smaller. I think the community is more than happy to give recommendations for libraries for specific tasks, but obviously it would depend on your situation whether those libraries do enough of what you need, or if it wouldn't be worth it trying to code the rest of it yourself.

Baggers

Again, you will need to define what you mean by 'modernize'. The reason I'm asking is that I'm writing GPU code in lisp, and that feels kind of modern. But I think the kind of modernization you are asking for might be the kind that would make me ditch CL entirely (more strictly OOP or more strictly functional... more strictly anything really).

Aidenn0

The problem isn't that nobody wants to modernize CL, it's that the effort is spread diffusely across a small number of people.

The other problem is that the language is so malleable, that you can easily do an 80% solution that solves just the problem you have right now, so there is less of an impetus to standardize on a complete solution.

dzecniv

See again cl21. And maybe join the effort on Corvus (a "low-level lisp for LLVM")?

phoe

I have no idea how to answer that one in general - that's a question that you, or anyone else, must answer yourself. Do you like the language? Do you find it fun? Do you find it productive? Do you find it suiting your needs? Do you want to spend your time here or would you rather spend it elsewhere?

There's no big bearded guy that's going to tell you "you're a good person/coder/programmer if you're using this or that language". All I can say is, the two years of my life that I've spent as a Lisper have been years very well spent programming-wise.


On-boarding


Why is on-boarding new developers so difficult? For instance, why isn’t Roswell linked to from every About Common Lisp page?

Baggers

Warning: Very much my own opinion

  • CL is enjoyable to learn with a good environment
  • Emacs/Vim are the only places with good environments
  • Learning Emacs/Vim and CL at the same is super hard

When atom-slime is stable I'll defintely try using that to teach people but currently (last checked 1 month ago) it's way too unstable. Please don't recommend it yet.

Roswell isnt recommended by most as most people don't use it. I'm certainly in that camp too, once I had a way of getting set up that was simple I stopped looking for other ways to do it (I made a youtube video on this process which has been fairly popular).

Also teaching is hard and the longer it has been since you struggled, the harder is can be to recall what made things difficult. The 'churn' of people in the community feels low and so it seems that many of the people around have been in for a very long time.

dzecniv

I'm not sure the community is aware of this difficulty. I hope that the CL Cookbook help in fixing that.

For me the difficulty was not about Emacs that I was familiar with, but the extreme lack of good, up to date and enjoyable resources. Do you imagine that many projects don't even have documentation (uiop points to its sources, series...), don't even have syntax highlighting for lisp snippets (common-lisp.net), is old and clunky (CLHS), that lots of stuff doesn't point to modern tools (all the old books) or is outdated? :( Hopefully we'll keep on counting on the cl-cookbook and the awesome-list, which have quite improved lately.

KDallas_Multipass

I would say that there are more books that provide value to the CL community in spite of being old than there are books that are way outdated, due I think largely to the lack of change in the standard. I too thought that the CLHS seemed clunky, and there are parts of it that I don't understand, but for the most part when I need it I put my cursor over a symbol and get the clhs page on the symbol and I'm good to go. It took me a while to find the section where loop's semantics are explained rather than just the grammar, and I go back to that a lot. I imagine there are other such references that I haven't found yet. Regarding up-to-date resources, I found PCL to be a great start, it shows you how to get real stuff done. The only thing that might be out of date is the practical about parsing mp3's and streaming them, and thats only because I have a hard time believing that people have libraries of mp3's sitting around anymore, or maybe thats just me. I also recommend Edi Weitz' cookbook for an example of more modern excursions, esp w.r.t. the section on concurrency, where he walks through using a library that lets you specify a graph of computation steps, allowing concurrent operations to proceed when called for, and synchronizing when necessary.

Aidenn0

Roswell didn't exist when many of the "About Common Lisp" pages were written. There are probably still some pages out there that mention "asdf-install" which is a failed precursor to Quicklisp.

Also, many of the advantages of Lisp development are in the IDE and tooling. This means Emacs if you want something stable. People say "I just want to program in lisp, not learn a whole new editor." This is a framing problem I think, because nobody complained about firing up TurboPascal for Pascal programming.

cannabis_detox

For people who feel they are great programmers and are just switching languages: PCL was designed for this purpose.

I can't speak for those guys. I was shit when I started lisp. I had used a lot of languages. I had made a lot of money off of software that I had written. After all, good code has nothing to do with good software. But I knew I was shit, programming was just way too hard. So I'm gonna talk about the people who were like me when I started out, and why they would hate and more importantly struggle to use lisp.

As I've alluded, I've been on my Lisp journey for a couple of years and I've gone from knowing nothing about lisp to the point where I can actually argue why I want to use lisp instead of languages like c# or python. To me this issue is pretty simple. Lisp's lack of popularity makes it much more difficult to glue things together in order to create software. You can do it, but it pales in comparison to how easy it is to do in Python, which is the best at this in my experience. If I want to write a program in python, I can google how to do it and you bet your ass there will be a library that already does everything I want to do (and more). In addition, I'll see at least 2 stack overflow questions on "how to do x". I'm not saying you can't glue things together in lisp, you absolutely can, and I think lisp actually simplifes so many things, but there is undeniably so much more in your way if you're a newb just trying to make something.

Let's get concrete! Lisp's standard library (despite being massive) does not provide you with enough to make a basic application just by gluing some high level functions together. Heck, even writing a file is a fucking mess. (defun write-file (file string)) pls. Imagine you know the filename and what you want to put in the file. That should be enough. Nope. You've gotta copy paste this mess that has something to do with something called a stream (whatever the fuck that is). Manipulating strings? Forget it! (regular expressions?! Now we have 2 problems! Lisp doesn't support regex out of the box? Make that 3 problems!) I haven't even mentioned the parenthesis. Programming lisp without rainbow parentheses is a surefire way to give up on it. If you are not a proficient programmer (ie the way I was before I learned how to apply recursion. ty little lisper), rolling your own functions to do string manipulation is a huge vacuum of your time and energy for something that you feel should already be there. Of course, once you understand how to program, it's easy to roll your own, and honestly when it comes to text, I've been thinking about it all my life, I don't need to reinvent how I think about it. I just need the computer to understand me. So if anything, I'd rather have my own functions that match my thought process instead of trying to reshape my mind to conform to regular expressions or the standard way of manipulating strings that everyone uses so you have to too. Lisp is so much about getting the computer to come to you instead of the other way around. But this is pointless when you're just trying to make something. When you're new and you're just trying to make something, it just feels like, "Why isn't this here? Doesn't everyone have to use strings? Why hasn't someone added this shit yet?" It's so frustrating that the only way someone is really going to put up with all that is if they really believe there's a payoff in the long term. Luckily, there is. Lisp rewards deep study. For example let's look at code substitution. Code substitution is great. It allows you to choose your inputs which is a very poweful thing. However, code substitution where your code is in a stupid simple data structure and you have a huge library dedicated to using that data structure, well that's not great, that's fucking beautiful. But you don't learn that in the same day. You learn about code substitution. You learn about manipulating lists. You realize that your code is a list (multiple times). And piece by piece it all comes together and you can start to see the tapestry those smug lispers wouldn't shut up about. It's fucking beautiful but it's not something you get any value out of when you're first starting out and just trying to make something. Lisp also requires blind faith and trust. You will benefit immensely from suspending belief, doing things the lisp way, and asking why do lispers do things this way. Why do we use let? There must be a good reason. Use it. Ask why. Trust that there's a smart reason for doing so. There is. And if you can't find a good reason, great! You're using lisp, do it your way instead. You're not forced to do things the lisp way. And I didn't even talk about development environments. There's just too much there, but basically it's another one of those long term rewards. I wrote my own toy IDE with LTK in ~30 lines of code that I didn't even realize could update itself in real time. So it's not even about learning EMACS SLIME. The language itself makes it easy to make really great development environments. And when you think about it, shouldn't every language? But when you're new who cares. just give me the IDE. Give me the libraries I need. Let me make my application. And suddenly all of these "features" I bragged about just stand in your way. None of that talk about beauty and reward matters to a gal who just wants to automatically download new episodes of one piece. The only thing she's thinking is why is this shit so pointlessly complicated and difficult. Even if you were to fix all of the problems I mentioned, you still can't compete with googling "how do i do x in python".

blue-dog-blues

My journey was a little bit different. I have always loved computing history read up on the exploits of ancient engineers as much as I can. Out of that reading I came to appreciate the following:

  • The beauty of the Lisp programming language.
  • The beauty of the APL programming language.
  • Just how unbeleivably smart the designers of Lisp and APL were. We are not talking normal smart here. We are talking 100 years ahead of the curve smart. Orders of magnitude smarter than smart people. Then consider that McCarthy and Iverson were not the only intellects of that caliber in the computing field in the 50s 60s and 70s. Computer science was bloody full of them!
  • Just how large of a canyon separates the everyday smart people in comp sci from the true geniuses like McCarthy and Iverse et al. Holy crap. Watching the smart re-invent over and over again some random subset of ideas already inventet so long ago is truly a thing to behold. It's eye watering.
  • Just how little we have progressed. Computer science has not really progressed much in the last 30 years. Most of the fruits of computer science that we use today were invented in the 50s 60s and 70s. All we do now is more of it and faster. Instead, to make sure the hardware folks get the well deserved spotlight, all of the innovation seems to have gone to hardware (not just better, faster, smaller and more of the same BUT new as well!).

So, for every complaint about Lisp my mind just scrolls back to any of intellectual giants of the past in the field of computer science and thank my lucky stars I was born after them to be able to enjoy the fruits of the genius. Because, what would programming look like without McCarthy? Would someone else invent Lisp or APL without them?

I think the world would be exactly as it is today, only a little bit poorer becasue we would probably only have a subset of the ideas discovered by these people. Everyone would be programming in some language that is essentially a cobbled together hot pile of garbage ALMOST ALWAYS containing only a subset of the ideas expressed in Lisp. A language invented 50 years ago for crying out loud.

The resulting conclusion though is that, there are truly two different demographics exploring Lisp:

  • The pragmatist - need to deliver a real product yesterday, don't have time for bullshit, gotta pay some bills. Real world economics rules the day, real programmers deliver product.
  • The admirer - real world economics either don't matter or have bypassed us and we spend our time in magical fields sown for us FOR FREE by giants that are either long gone or slowly passing away.

I preffer the latter than the former, perhaps one day we will manage the economics side of things to make a pragmattic Lisp experience have a lower barrier to entry.

Lastly, I never look down on the pragmatist. They have delivered 99.9% of the worlds code, not us. I am just glad I don't have to suffer their toil. Lisp makes it fun.

phoe

The documentation and materials are scattered and not that easy to find. I'm trying to fix that with the UltraSpec.


Why hasn’t the Common Lisp Foundation updated the common-lisp.net home page since 2015?

phoe

I have no idea. The Foundation has been alive in general, but it seems thay have other priorities than that. Why don't you ask them? #common-lisp.net @ Freenode.


Community


Why are there so few instructional or advocacy bloggers?

Aidenn0

I think it's because it's a smaller and older community. Also the community is very diverse for its small size.

phoe

Oh come on. Paul Graham is enough when it comes to advocates. :)

I think it directly has to do with the size of the community. Nonetheless, Planet Lisp has a rather large list of blogs that it scraps for articles. You might want to take a look at it.

Hexstream

The best kind of advocacy is making things better, which I'm already trying to do.


Why can’t Google search find many good resources? I.e. a search for “Common Lisp” is not that helpful.

phoe

Size of the community, again. I think one of the best places for finding resources is actually going to talk with people who use the language, say, {#lisp,#lispgames,#lispweb} @ Freenode or the Lisp Discord server and ask them for advice. The language is old, websites tend to fade along with the users and become stale and not updated.

Hopefully the UltraSpec (oh gods when is he going to stop talking about it) will alleviate this.

edgar-rft

Beause there are very little good resources of anything at all in the internet. Most other things are much shittier than Common Lisp, so nobody there notices. That's the only difference. It's a perceptional illusion.


Footer

More links for newbies:

  • http://lisp-lang.org/
  • http://quickdocs.org/
  • http://lispcookbook.github.io/cl-cookbook/

If you'd like your own answers to be included here, give me a hint on email.

The comments have been scraped and posted here in good will; they belong to their authors and I'll remove any of them the moment I hear from the author that they should be removed.