________________________________________________________________________________
I was a total raving Python evangelist for the first 12 years of my coding career, and then got a job as the CTO for a Python based startup that had been running absent a technical leader for 5 years, with relatively junior coders making all the decisions. I still love Python, but I now have a completely different attitude to hyper-dynamic languages (like Python, Ruby, Clojure, Elixir, etc). In my _new_ opinion, they are great for anyone making small projects, and good for disciplined, experienced teams making larger projects, but are really double-edged swords in large projects. If your team really knows what they're doing and can take advantage of the flexibility without creating a mess, they can let you move really fast and make elegant DSLs and so on. But if you let a team of juniors do whatever seems like a good idea, with nobody calling the shots who understands tech debt and the large-scale architecture problems, the mess that can be made is staggering. I never thought I'd say this before, but I would have been happier stepping into C++. :-/
Sure, this is a problem of people, not language. But there is something to the argument that absent discipline and experience, these can be dangerous. One can detangle a dog's breakfast in Java Spring a lot more easily.
(ever seen "import gc" in a Python program? yes, that means what you think it means..)
This is my experience as well. Incidentally "this is a problem of people, not language" is one of my pet peeve statements (not that I'm mad at you, of course!) specifically because I've had it used against me enough to defend choosing, imo, bad tools. Everyone thinks they have developers that are good enough that they don't need the bumpers up when they bowl. But honestly, it's just almost never true (and if it is, it won't be for long as your company grows).
One company I worked at had "we hire the best!" as its slogan, and they used it to justify so much bad practice. No code reviews (we just don't hire people who write bad code), no unit testing (same reason), python used for huge sprawling 10+ year old codebases that's on its third or fourth generation of maintainers, and no documentation anywhere. Everything is a spelunking expedition, and there's no type safety or anything to help you figure it out. There are more than a few thousand-plus line functions that make PyCharm chug every time your touch a character.
On the flip side, the Java and C++ codebases were similarly abused (no tests, no docs, four generations of maintainers/oral tradition, no code reviews ever) but are at least two orders of magnitude easier to traverse because the type system keeps everyone honest across decades in a way that dynamic languages just do not.
I actually do due diligence on tech firms now for a job, and you're 100% right that the tradeoffs change dramatically when the company grows. Which doesn't mean that using the best thing possible for your expert coders at the beginning is wrong. At the beginning, you need every advantage you can get, and if you have expert programmers, give them the sharpest knife you can find! But yeah, once your company is hiring hundreds, or even dozens of devs, and is being sold every handful of years from one investment fund to another, the same set of tradeoffs no longer necessarily apply and .NET and the JVM look pretty good.
I'm writing some Java at the moment for an Android app, and it's just killing me after doing Scheme and Python, but I have to admit, if that start-up had used Java or C#, they might have been better off in the long run.
Hiring really good, disciplined people is super, super hard. My current thought is that if you don't have a super-star CTO who can attract and keep real A+ coders, _don't use advanced languages_. And if you do have that CTO, do it, but make sure you are architecting to make the rewrite in C# or whatever doable when you sell your company. If I were starting my own business now, I'd use Clojure or Scala or F#, and design things so that in 5 years new owners had a decent path to switch to C# or Java wherever they want.
> I'm writing some Java at the moment for an Android app, and it's just killing me after doing Scheme and Python
Android isn't Java though. It's some frankenstein contraption made from a mutated version of Java from a decade ago.
> I were starting my own business now, I'd use Clojure or Scala.
I personally wouldn't ever choose Scala for anything. It's tooling is horrible (Sbt is a special hell, and scalac is as slow as a C++ compiler).
I'm currently in a code base like you describe, scala core with a switch to Java. It's not fun. Were stuck with these legacy library and framework choices that don't fit into the Java ecosystem and it slows down development considerably.
> ... and scalac is as slow as a C++ compiler
Scalac is as fast as Javac if you don't overuse Scala-specific features which are known to compile slowly - implicits and macros. But it is unfair to use these features and than complain on compile speed or compare it to a language that doesn't offer similar feature set.
Also, in our Java/Scala project it turned out that Scala+Bloop was way faster (10x-100x) at incremental compilation than Java+Gradle. Bloop is based on SBT libraries so there are some things that SBT does right.
Thanks for the compliment. But, as the author of Bloop, I must say that Bloop is definitely not based on sbt and it departs a lot from the design decisions sbt made.
Interesting, I've only read Scala. Personally I like Clojure but thought I should mention it as I see businesses mixing Scala with Java successfully in my work.
You can, but interoperability is not that great. It can work, but often it takes a lot of massaging/converting even basic data types.
Kotlin has wayyy better interoperability with Java
Android is the worst implementation of Java projects - horrible architecture or lack thereof and really bad api design. If anyone wants to learn Java, focus on modern Java language and micro services based frameworks and you can get best of all.
totally agree about sbt being gratuitously complex.
i wonder if Java is judiciously choosing the non crazily digressive parts of Scala, as they both evolve.
That is what Goetz said is the plan for Java. Move slow, let other languages experiment with language features, and then take the best features and implement them better.
Well, so far the “better” still remains to be seen... waiting for type-classes to represent monoids, functors, applicatives, monads and foldable.
So far I’ve just seen a complex and wild stream API that mimics it in practice, without all the mental model behind it.
Oh, and what’s this nonsense with Java Option?! Really?
We're yet to see if monoids, functors, and monads are worth it :)
For better I'd say you should look at things like virtual threads vs async/await or kotlins when vs switch expressions.
If you program modern Java or Kotlin map and flatMap are everywhere. I'd say the debate whether functors and monads are useful is over.
Java or Kotlin support functors or monads as design patterns, i.e. this is how a map/flatMap works with an Optional or a Stream.
The question is: if this support is enough or should the type system be enhanced so you can abstract over/compose a functor and/or a monad ?
The answer to this question is IMO still open.
Nice : as an ex Scala person for a few years, seems smart.
Sbt optional right? Can use maven or whatever else of you want.
Please use Kotlin on Android. Java on Android is a de-fanged and lame tiger with half the modern standard library and language features missing. Blame both Oracle and Google for this.
Having finished my first little app for Android in Java, I will certainly be switching. :-)
Androidifed Java is the worst thing you can deal to be honest.
Can you just use Kotlin to avoid Android's Java, or are there instances where you need to dip into Java get stuff done?
Kotlin is like a tape on the ugly Android architecture, little helpful but still a monstrosity compared to iOS.
ok that makes me feel better. God it's awful.
Why would you make technical decisions based on what hyponthenictal new owners would prefer in a hyponthenictal acquisition in 5 years?
Because that's how founders make money. The vast majority of exits now are purchases by private equity firms, and they will do a thorough technical due diligence and look at those decisions and what it will take to run the company the way they want to. (I do those diligences for a living) If you're doing a tiny lifestyle company with no exit plan, you wouldn't. Which ironically, is exactly what I'm doing on my own time, so I use Scheme for that. :-)
Most exits are purchases by private equity? I had no idea - are there any references for that available?
Here's one report I used for a presentation. The quote of interest:
"Over the last 12 months, 75 per cent of the most active “buyers” of $100m+ technology companies have been PE firms. This is despite the fact that the combined cash balances of the “Big Five” tech companies is an eye-watering $330 billion. Today’s most active tech buyers are private equity firms like Silver Lake, Francisco Partners, and Vista Equity. Not Google, Facebook or Amazon."
https://ftalphaville.ft.com/2019/01/07/1546862666000/Guest-p...
One of the things that is not obvious to people coming from the VC world is that once the company has gone from VC to PE, it's likely going to get flipped several more times from PE to PE. At a guess, I'd say 3/4 of what I look at are one PE to another. Depending on the firm, they will sell every 3 to 7 years or so.
iain
What's "hyponthenictal"?
You used it twice, but it's not on Google...
It's a misspelling of "hypothetical"
> I'm writing some Java at the moment for an Android app, and it's just killing me after doing Scheme and Python, but I have to admit, if that start-up had used Java or C#, they might have been better off in the long run.
that’s a logical fallacy. chances are that if they used Java or C# they might be dead by now (ie it sucks but usually it’s a good problem to have)
I know what you're getting at, and I agree with the sentiment very often, most of the time even. But in this particular case, I don't think so, and believe me, I spent a lot a of time thinking about that....
So what are your thoughts?
Uh hard. First, maybe don't buy companies with bad technical debt, just start from scratch? And always prefer two experts to 10 juniors in the beginning even if you move more slowly at first. I don't know what would have worked well in that particular situation, but if I were to do it again and could influence the original team, I'd use Clojure. I believe the immutability and control of side effects would have been more of a benefit than types. That was where the real problems came from, dynamic ability to create out-of-sight side effects and alter data in unexpected ways all over the damn place. Clojure's immutability plus spec would have been way better.
> Everything is a spelunking expedition, and there's no type safety or anything to help you figure it out.
For any significant Python codebase, I make sure to put extra effort into precise and strict type annotations so I can rely on type safety. Sometimes after fixing all the statically detected type problems, my program works on the first try.
Python was strongly typed from the beginning, before it was statically typed. When you add a string to a number in Python, you get a type error. Python's static type system– which is formally part of the language and has multiple implementations– is an elegant and natural fit to the language. Unlike TypeScript, where the type system is artificial, and _restricts_ what you can do with JavaScript.
Language features that made it into ruby– like blocks and unless– were rejected from Python because they would have made it possible to create advanced DSLs, which is something the language maintainers aim to avoid– enabling the creation of arcane application-specific special languages.
PEP-484 is the static type system's name:
https://www.python.org/dev/peps/pep-0484/
(implementations include mypy and Pyre)
Exactly my feeling honestly. I was a fan of Javascript and Python a couple of years ago, and I strongly (and still do) hated Java for its verbosity. I really have to admit that, especially with large teams, a strongly typed language is definitely the way to go.
I don't feel confident when I program in Python, nor in Javascript exactly for the lack of proper typing. One could argue that if you know exactly what you are doing, types can even be ignored, which is kind of true - but they're there to help the developer and his team. I don't see any good reason to use Python over Golang, for example, if we only consider the language itself and the confidence that you get from using a statically typed language.
I've often found that starting some quick-and-dirty thing in shell scripts is great, but once I get to around 100 lines of shell script code, I'm better off switching to python (or similar). Similarly, once I get to around 1000 lines of python code, I'm better off switching to compiled, statically typed language (e.g., Java, C, C++, etc.).
So my own heuristic has become:
shell script when < 100 lines of script
python code when < 1000 lines of code
else statically typed, compiled language
I live by very suspiciously similar rules of thumb although my tolerance in the past for dynamic scripting languages was probably closer to several hundred lines of code.
However, I have to acknowledge that modern IDEs (and even advanced code editors like VSCode) have pretty good support for dynamic scripting languages these days that make working with them in larger code bases not entirely painful.
Yes, PyCharm was an amazing help.
I feel that Python has (among many, many other things) replaced Matlab and the likes.
These were the go-to "prototyping" languages people in science, R&D, etc. used (and still use!), and then you'd port those into more manageable languages for larger scale projects, or actual software products.
These days, Python does all that. Unless you need some extreme performance for your final product, a well-optimized Python program will work just fine.
But still, there's some of that matlab legacy left - instead of one comprehensive application, you have tons of smaller programs floating around, for one-off/single-issue uses.
On a continuing project with a company where everyone speaks Matlab as a second language, I succeeded in switching the guy writing Matlab code for the Matlab REST API to Python/numpy/scipy for the industrial analytics we need, saving us $25k a year in licence fees and VM costs.
This was my heuristic years ago before Python got type annotation support. Now there are linters that take type annotations into account, and LSP servers that can catch type errors as you write Python code.
I've found that using Python for large projects is both fine and productive as long as you utilize type annotations and modules. Ambiguity goes out the window when you do that.
I was super hopeful about type hints but they fell short of my expectations, unfortunately. It takes you to a place full of kluges and workarounds and hacks which is all .... really unpythonic.
In the end I feel, if you really want that, use a language that treats types as a first class citizen. There are languages with most of Python's attractive features but actual optional typing (we use Groovy), or static with really good inferred types (eg: Kotlin) etc.
I was really happy when I first learned of type annotations being added to Python. However, much of that excitement evaporated when I read that it was just a 'hint' and not something that's enforced.
This was also my initial reaction. However, if you couple type annotations with an LSP server[1] that takes the annotations into consideration[2], you'll get on-the-fly type checking in your editor or IDE. That way you can catch type errors as you write your typed Python code.
Besides VS Code, these days lightweight editors like Kate, Sublime Text, Vim, etc can all take advantage of type checking LSP servers for Python.
[1]
https://github.com/palantir/python-language-server
[2]
https://github.com/tomv564/pyls-mypy
It's not enforced by CPython the interpreter. You run mypy to enforce the type hints:
It's quite good.
I like it a lot. Use it with --disallow-untyped-defs to make sure more is caught.
Nice guidelines! Though in my experience, the problem with the python code of < 1000 lines is that it tends to grow when you're not looking, and suddenly you find yourself needing to maintain (or rewrite) a Python monster.
what a really great way to think about it...i've been doing the same and hadn't thought of it that way (for decades)
Some interesting responses here, thanks. Not surprisingly lots of people are saying "types would have saved you". I'm sure they would have helped, though I personally think they do introduce a high cost too, and I'm a lisp-head nowadays not a Haskeller.
The real culprits, thinking about it since my post, were _side effects and mutability_. Python lets you create incredibly difficult to trace chains of side effects and mutations, and has basically no decent ways to prevent this. In Scheme, my code is still super small and I can dynamically create all kinds of object like things, but if I want private, I can make it god-damned private. In Python, anything can be changed by anything ("we're all consenting adults") and using side effects in weird ways is actually part of the idiom. I don't know how many times in Python literature I've seen some variant of "you don't need those baroque patterns because we can use 'import' as a singleton, running class initialization as the constructor". And so all the frameworks have crazy thread-local magic happening from bloody import statements!!! Do that too much and you have no idea what's happening where and why, and something as trivial as changing the order of imports can kill your app. Where I was, this had gotten so bad that the app couldn't even be turned on and tested in the normal way, and none of my predecessors had been willing to go through the pain of figuring out what the chain of imports were doing to bugger it up. (Because that didn't look like doing anything productive, I'm sure you all know the drill...)
If I were doing it again, personally, I'd use Clojure and Spec, and worry more about mutability and side-effects than anything else. Just my two cents Canadian.
Thanks for making this point. People focus on types but it's the whole smorgasboard of modern features missing from Python - lack of proper (idiomatic) support for functional programming, None types, etc etc.
I don't disagree, but I will add that most long-lived Spring applications I have worked with were disasters. Figuring out how a Spring application actually works (or why it doesn't) can be difficult. Upgrading the framework, or switching to a different one without breakage is often a non-trivial task.
The best codebases I have touched, in any language ecosystem, either didn't use a framework, or kept the framework firmly isolated from the other code. If you build your code "inside" a framework, it will eventually become an obstacle to change.
Furthermore, the best Java codebases I have seen avoided annotation-driven-development, reflection, classpath scanning, DI frameworks, AOP, etc. Anything that feels at all magical should be handled with great suspicion.
> Anything that feels at all magical should be handled with great suspicion.
This. True in any language.
This precise struggle is why Go is created, and also why Rob Pike said what he said. The industry needs a statically typed language with good ergonomics (not typing too much) and performant.
I agree with you, asking everyone to be discipline with their code in a dynamic language is a bit too much to ask (unfortunately).
If there is something similar with Django on golang ecosystem, I would use golang more. Battery-included web framework seem to be out of style these days and no one invest in developing one for newer language like golang. As a single developer working mostly alone, microservice-based development is quite painful.
Gobuffalo is close but it's been a while since I've looked at it. What you said is has been my experience too.
Here I go about the batteries included vs micro thing. People pick Flask because it looks approachable but then DIY features in. The same thing happens with Sinatra+Rails. If the python ecosystem grows and other communities come in, maybe they will bring their culture and standards in.
Take poetry for python. It's basically, "hey ... cargo/yarn/mix/bundler all kind of figured out these ergonomics". Poetry's "why" section in the README really resonated with me. Cross pollination of ideas across tribes is _good_.
But then, I'm biased/blub-paradox of course. And it's definitely in line with the productivity/DSLs/rapid vs types/verbosity/slower modes discussion.
Not sure about golang but D language has a battery included web framework based on the excellent vibe.d [1].
Fun fact, D language forum website is probably one of the fastest and the most responsive websites on the planet [2], and it's written in D but not based on vibe.d or Diamond [3].
[1]
https://code.dlang.org/packages/diamond
[2]
https://news.ycombinator.com/item?id=16731302
[3]
https://news.ycombinator.com/item?id=3592769
Second this. There's no rails/express for golang. Personally I'd love to see a next-gen Rails-like framework arise in Swift. That would be amazing in my opinion.
Go needs to be renamed to something that can be googled.
Golang?
Yeah. When you have two experts in a garage, it rocks. When you absolutely need to hire some new folks in little old Victoria in the next two months so you turn to bootcamps and interns, not so much.... ;-)
Victoria, BC?
/me waves from Quadra (Island, not Street).
Not job hunting, but always happy to chat with people doing cool stuff sort-of locally.
This is just my humble opinion, but i'd argue that it is much easier to hire Go developers than Scala or Clojure, even outside Silicon Valley.
Too bad the lack of generics means it has awful ergonomics in reality.
I also think rust's ? makes the "return err" pattern much nicer to deal with.
I like generics because it actually solves a lot of things that OOP and inheritance were supposed to without turning your dependencies into a ball of mud.
After 30 years of mediocre programming things I hate are unplanned/hidden side effects, train wrecks due to mutable state, and out of control dependencies. Error handling at this point seems like Weltschmerz.
I have done big and small python/C/C++ projects. For python small projects it is pretty sweet. You type some commands and you are up and running. With c/c++ it seems there is always this 'big' first time step for setup. Yet long term that seems to flip around like you have seen. Where that first time setup works pretty good for a long time. Whereas in python the org of the program becomes this dead weight you are kind of fighting. It is not awful but just kind of there. Now that is just my exp I am sure others had the opposite. But it is nice to see someone else out there thinking like I do about this. python for nice small contained things. other languages for longer term bigger projects. But it is same problem most of the time you have in many orgs. The prototype becomes the production thing :(
People always ask me 'how do I become a programmer' the first thing I always tell them is 'you need to learn python it is fairly easy to pick up and learn' I aim them at the python org website and their excellent tutorials. Funny enough it weeds out the people who do not really want to be a programmer but thought it sounded cool, as they have to stop and learn something.
Yeah, I hear you. Nowadays I do Scheme embedded in C/C++. The stuff I used to love Python for is now done in Scheme, and the outer containers are in C++. It's actually a really nice way to write code, for me at least. Forces a clean architecture on you, and you know you can always move anything out into C++ if you need either a library or lower-level access. I use S7, but you could use Guile, Gambit, Chicken similarly.
This is an interesting architecture, can you provide some more details on how exactly that works? E.g. do you use a c++ library that interprets scheme?
yeah, I use S7 Scheme. It's a complete scheme interpreter in one C file, super easy to embed, quite similar to Clojure linguistically. Basically all "business" logic is in Scheme, all OS/UI interaction is in C or C++. For me, it's amazing, because I write music apps and I can reuse my model code across desktop, phone, in Max/MSP or PureData, in web assembly, everywhere. Being able to prototype in Max is awesome. My model layer _only_ knows about music, the high-level user actions (ie "play note", not "click button"), and Scheme. It doesn't know anything about widgets, phones, etc. All boundaries are crossed with the C FFI layer (which is really nice in S7) forcing a strict ports-and-adapters approach. There absolutely is upfront cost, but the liberation of being totally decoupled from any framework or vendor in my model code is awesome. I chose S7 because I'm doing music, but you could do similar with Guile, Chicken, Gambit, Embedded Common Lisp, etc. And Scheme is so minimal that if I needed to switch Scheme implementations it would not be a big deal - I would only be rewriting the adapter layer.
Interesting, one of the most successful companies I've seen in my tech due diligence work was doing the same thing in high end scientific computing. They had written their own DSL, and a whole containing application layer in C that could run on any target platform, with GUI adapters for windows, unix, osx, etc. They were doing _very_ well. And they could get amazing scientists to work for them super-productively because the DSL was designed around the scientists needs.
This sounds awesome, glad to hear it's working well for you.
One question that comes to mind for the business stuff is security. Basically, are you accepting Scheme code from untrusted users, or is it just a bundled part of the app?
The reason I'm asking is that that run-time flexibility can completely bite you in the ass if someone can submit code that does unacceptable, unanticipated things. For example, I had an engagement with a company to review their main application, that they host. The issue was that while the app was in Java, it accepted user templates in (IIRC) BeanShell. One quick System.exit() in a template as a proof of concept, and Tomcat came thundering down.
The concept still works of course if you restrict the language accepted to some minimal DSL.
Interesting. This actually sounds very similar to game developers embedding Lua or V8 for various logic. I've used Lua myself in a similar way, embedded in a high perf application, with a user script driving the logic.
Yeah Naughty Dog was doing the same thing in Scheme apparently!
>But if you let a team of juniors do whatever seems like a good idea, with nobody calling the shots who understands tech debt and the large-scale architecture problems, the mess that can be made is staggering.
I've seen the same occur with Spring based Java. To me this is similar to the old "no one has gotten fired for going IBM". The new phrase would be "no one has gotten fired for going with a static typed language."
When someone writes bad Java code, we tend to blame the developer. When someone writes bad Python code, we blame Python.
If you think it's the lack of types that make things more difficult to follow, you can always force build failure if type hints aren't provided (which Python has supported for a while now) in the same way you can fail your Java builds if test coverage isn't up to par.
To be specific, it's the lack of typing that makes large code bases in Python, Ruby, and especially JavaScript a huge pain compared to Java. It makes debugging much more difficult. Thankfully this is being fixed with stuff like Typescript and similar projects for both Python and Ruby.
Some people may argue, "but documentation would make this an non-issue". Guarantees built-in to the platform itself are more reliable than human aspirations, or automatic memory management as a feature wouldn't have gotten popular.
I actually found it was untraceable side effects from imports and ability to monkey patch that was worse. Related to the typing, but not totally the same. I would pick something like Clojure or F# now to control that. Side effects and coupling.
You can patch by linking a different library in C. What's the difference? I suppose it's easier in Python to patch. Seems like a good thing to me.
Didn't realize that monkey patching wasn't only limited to Ruby.
It tends to be frowned on in Python. Mostly because some fairly popular projects did it first, and if you monkey patch over them, you break them.
Consequently, there was a big disincentive to monkeypatching.
IMO I doubt switching language would help. It's the same for any language. You can learn to code easy, but learning to write good code/architecture + working together in a large team is really hard.. takes lots of experience and time till the team can come together.
Some languages and frameworks have the right defaults for scale.
Rails has a lot of magic that gets you running fast but it doesn't even have a service layer - and instead promotes the abomination that is mixins (concerns), fat models, fat controllers - stuff that gets you running with minimal effort but then crying over the code duplication all over the place and lack of logical separation.
ASP.NET is more verbose out the gate but you're basically guided in to stuff like repository pattern, service layer with POCO models, thin controllers and IoC. Static typing gives you guarantees when reading the code (I've seen RoR concerns that relied on random fields being present in target class, but had conditional logic with implicit assumptions about which class it would be included in - it was a hell to reason about). It's verbose but consistent and built to scale - I'll take a dirty ASP.NET project over a dirty RoR project any day.
+1. Every time we look at Ruby app in my tech diligence work, we wind up talking with them about how they wish they had that service layer and weren't doing Active Record methods from controllers. It's a real achilles heel of RoR once things get big.
You are not wrong. Any startup people reading this... your CTO is the second most important person at the company, maybe even the most important. They need to be awesome. You can't skip this.
> If your team really knows what they're doing and can take advantage of the flexibility without creating a mess, they can let you move really fast and make elegant DSLs and so on. [..snip..] I never thought I'd say this before, but I would have been happier stepping into C++.
Actually, I find Nim language [1] to be a great alternative to C++ as far as compiled, statically typed, high performance modern languages are concerned. Moreover, Nim's meta-programming capabilities (generics, templates, macros) are second to none for creating elegant DSLs. And its syntax makes Python developers feel at home. Preempting questions about garbage collection -- Nim allows to switch off GC and manually manage memory if a programmer so desires. Or you can manually manage memory in some critical parts of your program while letting the rest be garbage collected.
[1]
> But if you let a team of juniors do whatever seems like a good idea, with nobody calling the shots who understands tech debt and the large-scale architecture problems, the mess that can be made is staggering.
In the several years I spent doing .NET consulting, I believe that this is the crux of the problem, and not Python per se.
In .NET land (probably Enterprise Java too), this often manifests as an apparent goal to get every single pattern from PoEAA [1] into every file rather than extensive metaprogramming - but the root cause is the same - juniors not having appropriate guidance.
[1]:
https://martinfowler.com/books/eaa.html
I was about to say, I'd like to know which language juniors don't make a mess of.
> Sure, this is a problem of people, not language.
I don't think so. It is a problem of language.
Or more specifically, of dynamically typed languages (DTL) vs/ statically typed ones (STL).
We know for a fact now that DTL's simply don't scale to large code bases. And before dozens of people tell me there are a lot of large apps written in DTL, this doesn't mean it was a good idea in the first place nor that these large apps are easy to maintain and improve (they're not and they would be much easier to maintain and improve with type annotations).
There are simply too many large projects done in dynamic language to show that it's probably a good idea for them.
Or they eventually come up with a hybrid micro services.
> We know for a fact now that DTL's simply don't scale to large code bases.
We don't know that. This is not a fact.
You know I would agree as someone who did a lot of python and dynamic languages there is a tendency for chaos. But then I have seen just as many chaotic Java code. If even go so far to say it’s more a function of the libs/framework than the ecosystem. I have seen a lot of terrible flask code, django code was much better on average and often better than spring code.
_(ever seen "import gc" in a Python program? yes, that means what you think it means..)_
I recently spent half a day implementing a hacky workaround for something using gc. Once I was happy that I got it working, I immediately laughed at how absurd it was and deleted it.
I still don't think I will ever be happier in C++ but I'm only 11 years into python, and 6 years into coding full time, so who knows :-).
> I still love Python, but I now have a completely different attitude to hyper-dynamic languages (like Python, Ruby, Clojure, Elixir, etc).
Elixir is all about immutable structs and pattern matching. I don't think "hyper dynamic" is a fair characterization at all.
To be fair elixir has slowly become more and more static over time, as in via convention and tooling.
Even at the language level, every single variable is a constant—the opposite of "hyper dynamic".
The larger design patterns build on that immutability on the local level. Inheriting the design patterns from Erlang/OTP also make larger programs much more feasible. I can be away from a piece of Elixir code for months and come back and the structure is usually pretty straightforward to pick up again. Primarily due to supervisor hierarchies combined with the lack of weird OOP hierarchies and entanglement.
I have seen plenty of C# projects with these same problems.
I've seen plenty and worked on plenty too, but IME a lot of those issues were at the boundaries of the type system:
Abuse of untyped ViewBags to pass data to views, using ViewBags to pass things to Helpers, badly configured AOP or XML-based dependency injection, people using Dictionaries/dynamic/JSON instead of objects for passing data between application layers...
As soon as you have escape hatches you get those things that make refactoring a nightmare in large projects and cause new employees to always fail to grasp the full-picture because of how fragile projects were.
In the frontend I feel the same about Vuex. It's a great library, way more ergonomic than Redux and the defaults are great, but the fact that actions and getters are untyped by default in Typescript is a major source of bugs in projects I work on, even in codebases with 100% coverage.
But even static types (even the Hindley-Milner variety) won't be of much help in a team of juniors with no technical leadership. The codebase will end up in the same place - needing a rewrite - because even a good typing discipline cannot immediately substitute hard-won lessons in program design.
In fact I never thought I'd say this - for the first two years of discovering sum types, I thought they were the silver bullet of computing, and certain experiences have sadly tempered me there :)
I still agree that types _can_ make the flow and transformation of data in the system clearer than if there were no types. Though without experience, we could very well end up with convoluted designs and that can be as difficult to untangle as a dynamic spaghetti.
If nothing else, a statically typed mess is easier to navigate, understand and refactor than a dynamically typed mess. Which I think is a big deal.
Use the right tool for the job. Python is great except at the huge (or performance) end. The problem occurs when one tries to use it for everything. And it has tools now to help at the high end, though you have to choose to use them.
I have found in my own work life that the problems you get from large python code is quite different from a large C code. The python code usually start out very good and readable but as it get extended and changed it tend to create a massive mess that is hard to navigate. The C code in comparison don't generally get extended or changed at all since the initial work to even add the simplest change tend to require a large initial dive into the design, custom types and design details, that such code tend to mostly be left alone.
The upside with the python code is that a complete rewrite of parts that has get extended beyond recognition tend to be easier than expected.
> The C code in comparison don't generally get extended or changed at all since
That sounds like a disadvantage tbh
> If your team really knows what they're doing... they can let you move really fast and make elegant DSLs and so on.
I agree on this part. The problem is many teams with 5 Ruby programmers is doing what 100 Java programmers' work. There would usually be many teams in Java projects, and nobody have full picture of the project. Then people would create jobs out of nowhere because they have to do something... Just see how many projects have dedicated teams writing piles of code for Kubernetes.
I would rather hire much fewer elites for higher price instead.
My experience as well. I was a Perl then Python/R evangelist. But I doscovered and have been using Kotlin (JVM) extensively and TypeScript to a lesser extent for the last 2 or 3 years. And the more I dog and learn about the JVM the less I want to go back. I'm also interested in getting back to Haskell, Scala and Rust I toyed with just to keep my mind open.
If you have junior engineers you can't let them run willy nilly, you have to make them slow down, come up with a design (obviously I'm talking about >$medium_size projects), have a senior engineer review it, and then let them go. They will do much better and have more confidence in their work. Obviously you have to follow up weekly. They're junior engineers for a reason. Even a language like Rust or OCaml won't keep them from effing things up.
Programming languages don't kill projects -- _people_ kill projects.
Have been on a similar journey within the company I'm at now. Starting out with fairly loose Python, and now seeing egregious bugs make it into production and hurt your team's reputation, you start asking "why the hell are we suffering from things that are highly preventable". You work your way through type hints, strict linting, code reviews, etc etc, but you end up realising you're in part just using a hammer to nail in a screw.
I feel like it's nothing to do with the language. The quality of the team will reflect on the quality of the codebase. Period. Quality of teams/codebases is always going to follow some type of distribution.
Engineers are perpetually keeping mental tallies and notes in search of data on which tools, methods, and technologies will help them reach the promised land. I think really the majority of that is noise.
Not surprising. Lisp Curse has its own way of creeping in, but then again I'd take this any day over cubicle-farm software that is so sterile and stuck up that you need a 2 week long tutorial in order to run anything (I'm looking at you Google with your fancy-schmancy Borg and other inscrutable infra).
http://www.winestockwebdesign.com/Essays/Lisp_Curse.html
> (ever seen "import gc" in a Python program? yes, that means what you think it means..)
I'm using it sometimes, what's the issue?
I've used 'import gc' in python programs. gc.get_referers and gc.get_referents is very helpful.
I'm guessing you've been doing this for a while though... when you see that in a code base that you know has major issues and was made by fairly new programmers, it's like "oh god please tell me that doesn't mean garbage collector, I'm going to need a stiff drink"... :-)
yes, I have been doing 'import gc' for 20 years. Well, technically, I stopped about 15 years ago because I haven't had any unexpected references hanging around for that long.
gc.get_stats() too.
> ever seen "import gc" in a Python program?
Yes, I see it all the time. People think that `gc.collect` collects all the garbage, but that's not true. It only checks for reference cycles.
Doesn't C++ have the exact same problem? People have style guides that prohibit using part of the language features. Maybe the problem is just better understood in C++ land.
"import" as a replacement for a singleton, such a loaded gun. :-(
On a large software team, about half the programmers are below the industry-wide average. Sure, you can carefully build a team of 10 where they're all above average. I've even seen a team of 30 where most were above average, and none were clearly below. But a team of 100? That's a lot harder.
This has serious implications for how you develop large systems. Don't try to get too sophisticated on a larger or longer-lasting project. You'll have people on it who won't' be able to play at that level, and if you try to make them, they'll make a mess.
The same thing happens over time. Sure, your team is only 20 people... today. But if the program lives for three decades, how many people will work on it? 100? Will they _all_ be superstars? Even those maintenance people you hire 20 years from now? Probably not.
> but are really double-edged swords in large projects
Yeap. I ended up dragging my team to Golang due to this. Sure they didn't write unit tests and they still aren't writing unit tests. But at least now a compiler will take a look at the code before it's run. Lots of stupid errors at runtime disappeared.
For personal projects? Something like Python, Lua or Scheme is great. It can still be great in large teams, but they need to understand and care about what they are doing.
> but I would have been happier stepping into C++
Oh no you wouldn't. That's the nuclear foot gun.
I've been tasked(along with some great engineers, most more experienced than myself) to fix a mess left by two other teams. A project that was severely over budget and very late.
It was the worst abomination ever known to man. Crashes in random places. Race conditions. Global variables. Memory corruption galore. Lack of understanding of C++ in general (copy constructors, assignment operators, initializer lists) leading to many more bugs. Leaked like a sieve.
A lot of this wouldn't be possible in other languages (memory corruption in general). Some issues would happen no matter what (two modules opening the same file and writing on it simultaneously), unless something like Rust or Haskell was used to make this more difficult.
So far, that's the only project I've contradicted Joel Spolsky and it was completely thrown away and rewritten. Again in C++ because that was the requirement, but with proper modules this time, smart pointers and the like. It was completed in 3 months.
ok you're prob right. that sounds bad. :-)
> It was the worst abomination ever known to man. Crashes in random places. Race conditions. Global variables. Memory corruption galore.
All of these are trivially detected with msan, tsan, lsan and ubsan.
Absolutely agree with you. But with advent of microservices architecture patterns, it is possible to keep service code within bounds of maintainable size.
I'm by no means an expert architect yet, but has the argument for microservices in _smaller_ orgs basically become:
"Since y'all can't seem to use classes and interfaces correctly, let's physically separate the software so changing the service contracts becomes a bigger pain in the ass"?
I'm not even trying to be sarcastic. I'm genuinely curious. I feel like 90% of the benefits of microservices would be accomplished with discipline and good code review, but those things of course are more easily said than done.
(I understand your Facebooks and Netflixes benefit from having completely separate teams responsible for services with more autonomy, but I'm talking about small to midsize orgs that seem to have adopted microservices.)
Microservices are different from language abstractions (classes, etc.) In two important ways:
1. More decoupling means you can use an entirely different langauge
2. Services can be deployed, restarted, etc. independently.
But that comes with disadvantages. The contract then becomes message passing and serialization/deserialization, which is fairly primitive. You can't pass functions/callbacks, and simple static checks become a whole validation problem. It also limits your ability to use interesting types across services.
The XML craze 20 years ago tried to tackle all of these problems (remember XML schemas?), but it became unweildly and didn't really succeed despite a huge enterprise push.
In a mid-size shop is another language an advantage?
It certainly can be. Sometimes you start in one language but then need to integrate with something else that really only works well with a different language.
The more I do this, the more I come to believe that boundaries, no matter how you do them, are the million dollar question.
Or it's the other way round.
In certain languages, developers may find it difficult to maintain code quality and good design in a codebase of a certain size. So they find other ways of achieving good design by introducing service boundaries.
My experience is that, unless I am getting something extremely powerful with OS-like aspirations (Common Lisp, SmallTalk, Forth), I'd rather have static typing in a professional setting as well.
So a hierarchy of choice for me would be like:
S The languages mentioned above
A Languages with Hindley–Milner type system and inference (SML, OCaml)
B Your typical blue collar languages (C, Pascal, Go, Java, C#)
C Scripting dynamic languages (JS, Perl, PHP, Lua)
This is true for any language, strongly typed or otherwise, with or without memory management. The evidence gathered in the last decades is pretty clear: there is little to no advantage using a strongly typed language over something like Python or Ruby (on mobile so can't look up references but there's really plenty). And that's just looking at bug density, without even considering how quickly you can build something (often thanks to the great ecosystem of packages) or how easier it is to hire decent Python/Ruby/JS devs over, say, C++, Java or Rust.
That said: I wish you the best of luck!
The smoking gun wasn't really the types to be honest. It was the way you can monkey patch and have massive daisy chains of effects from a simple import statement. Import in Python can pretty much be made to do anything you want...
But Python is strongly typed. If you say x=2, type(x) will say “int”.
It does suffer from an excessive overloading of operators (“a” times 6 shouldn’t work). But that’s neither here nor there.
If Python wasn’t strongly typed, it wouldn’t be conceptually possible to move it towards _explicit_ typing as the current push is.
Now, it’s not _statically_ typed. You don’t get to assign values and interpretations to memory addresses. This is what “performant” people usually defend. But _that_ has nothing to do with typing assuming semantic roles in codebases.
Apologies for the "flexible" usage of the words "strongly typed". Arguably (as someone already pointed out) Python can be considered strongly typed (i.e. doing 1 + "1" won't cast the first operand to string, unlike e.g. JS) although there doesn't seem to be a clear definition of what strongly and weakly typed means (
https://en.wikipedia.org/wiki/Strong_and_weak_typing
). So let's stick with "statically" typed, which in the case of Python would mean using something like mypy.
Here are some references (there might be some overlap):
-
https://labs.ig.com/static-typing-promise
-
https://danluu.com/empirical-pl/
-
-
https://medium.com/javascript-scene/the-shocking-secret-abou...
-
https://www.researchgate.net/publication/259634489_An_empiri...
(one of the original studies)
I hope it's clear I'm not implying that there are no advantages in using a statically typed language, only that it's often seen as a solution to a problem that doesn't originate there.
PS: what is up with the downvotes? What is this, Reddit?
> The evidence gathered in the last decades
Source?
I was a Python dev for 14 years or so (now doing mostly personal projects in C/C++, Scheme, Clojure), and 10 years ago met my partner who is a biologist. I think one thing programmers underestimate is how many scientists and academics are using Python. It's not just ML, it's pretty much every kind of scientist and tons of non-science academics too. The fact that it's a great "casual" or part-time language is huge for these people. They aren't full time (or even self-identifying) programmers, but their programs are non-trivial. The readability, portability, tooling, and massive package ecosystem is huge for them. In that world, being somewhat comfortable in R and Python is becoming table stakes for really large swathes of academia.
Personally I believe this is exactly why Python is dominant in ML now. We are harvesting after 20 years of academics happily writing libraries and tools for academic work in Python. :-)
So I have no idea how the article got their numbers, but I'm not at all surprised to see the sheer number of people programming in Python exploding.
I'm one of those scientists, working in industry R&D. I use Python for everything from Jupyter notebooks to a roughly 8000 line program that runs a prototype of a product. Python pulled me away from Visual Basic several years ago, and then I have a longer history going back through Turbo Pascal and ultimately originating with primordial BASIC. On the hardware side, I use C within the Arduino ecosystem. I've got roughly a half dozen other scientists at my work site using Python at various levels.
Confession: Code quality is a problem. I care a lot about code quality. My team recently brought on a couple of people with real programming background, and I've asked them to help me bring myself up to date on good techniques, and it has helped. I was lucky that some of my former training, e.g., in Pascal, was not just about how to program, but actually how to write good code (for its time).
But here's my view. Don't just tell us that our code sucks. We already know it, but we're under the same pressures as anybody else, and programming is a force multiplier that we won't give up. We need instructions that we can understand at our level, that will help us do this stuff better, perhaps incrementally. Starting with a strict discipline is hard, but Python supports the ability to go back through working code and improve it.
This is an area I want to get more into actually. I've done a bunch of freelancing for scientists, and it seems to me that there is now a wave of scientists hitting the code size and complexity level where understanding the principles of maintainable software architecture is become pretty important. One of the gigs was reverse-documenting some RCPP because they had been left with a ball-of-mud someone had to write in a hurry and no one knew how to break it up. It was interesting, and cool to help people doing real science. If you have any thoughts about the viability of teaching architecture to scientists, I'm all ears. You can find me on linked on or my user name plus the email of a behemoth starting with G... ;-)
Your post reminds me of a segment[0] of a Pycon keynote where Jacob Kaplan Moss talks about "real programmers". One of the anecdotes was that he was hiring programmers and he stumbled on a student who had worked on a project who told him she can't apply because she "isn't really a programmer".
>_I've asked them to help me bring myself up to date on good techniques_
Here's an old draft of our onboarding document[1], and a few comments here on HN about this[2].
PS: in case anyone is wondering how I found the segment in question, I use a Chrome extension called "YouTube Captions Search"[3] went to the video, activated closed captions, clicked on the extension's icon, and searched for "geographic" because I remembered that the system the student had built was a geographic information system (GIS).
- [0]:
https://youtu.be/iOp0mSDCh6c?t=1084
- [1]:
https://jhadjar.gitlab.io/kbase/hiring/#resources-to-learn-o...
- [2]:
https://news.ycombinator.com/item?id=15576496
- [3]:
https://chrome.google.com/webstore/detail/youtube-captions-s...
I'm working with some researches on about ~40k lines of Python disease modelling code and these are the things that helped us becomes more productive:
- tests (unit, smoke, acceptance) (eg.
https://mattsegal.dev/alternate-test-styles.html
,
https://understandlegacycode.com/approval-tests/
)
- continuous integration (via GitHub actions eg:
https://mattsegal.dev/pytest-on-github-actions.html
)
- 10x performance improvements via cProfile (
https://julien.danjou.info/guide-to-python-profiling-cprofil...
)
- getting fast feedback on code changes by visualizing outputs with Streamlit (
)
- input parameter validation (eg.
https://pydantic-docs.helpmanual.io/
,
https://mattsegal.dev/cerberus-config-validation.html
)
- total automation of data processing job (eg.
- we don't use this but that's the kind of thing we have)
As to how they get the numbers, this is TIOBE's methodology: [1]
Since there are many questions about the way the TIOBE index is assembled, a special page is devoted to its definition. Basically the calculation comes down to counting hits for the search query +"<language> programming" In the next few sections it is explained what search engines qualify, what programming languages qualify and how the ratings are exactly calculated. Search Engines There are 25 search engines that are used to calculate the TIOBE index. The selected search engines are the 25 highest ranked websites of Alexa that meet the following conditions: The entry page of the site contains a search facility The result of querying the site contains an indication of the number of page hits The results should be available in HTML with clear tags Search engines in languages with special characters should be encoded properly The search engine should at least return 1 hit for 1 query The results of querying the site shouldn't contain too many outliers Porn sites are excluded
[1]
https://www.tiobe.com/tiobe-index/programming-languages-defi...
> Chaturbate.com: PORN_SITE
Gave me a laugh. Engineers would initially qualify a porn search engine to search for this sort of experiment
That'd certainly give "Python" a leg up as a search term.
Well, yes. Why wouldn't you? They hire engineers, and they have programming projects. Meets all requirements.
From what I've seen, jupyter notebooks reach convoluted excel formulas-level of accessibility and efficacy for lay people.
Most of us really underestimate what "non" programmers can achieve with if/else, loops, equality tests, a repl and autocompletion.
Yeah I'm guessing code quality is _tanking_ around the world. Lots more code , a lot less quality.
Perhaps, but when I built the shed in my back yard I didn't enforce the same standards a construction engineer would for a skyscraper. I increased the quantity of "shitty construction" in the world, but it works great for my purposes.
And a lot more stuff being done. Cut it out with the elitism.
It's like being an anaerobic bacteria complaining about the abundance of oxygen in during the Cambrian explosion.
"Back in my day no stinking, puny, oxygen breathing things were around, only sturdy creatures such as myself" :-)
Yeah, it was messy, but that's how we were able to have humans, hundreds of millions of years later.
Not every programmer is a "software engineer" for many, code is a tool to get a job done, it's a means to an end, and that's wonderful.
What is "code quality" anyway?
Just want to bump this from another perspective: I’m an engineer in a big EU science infrastructure project and everything has a Python API. The main lab environment is JupyterLab. No one wonders whether what languages scripts or workflows will be written in.
Julia is on the radar but even in Julians’ wildest dreams Python remains relevant through PyCall. Why rewrite it if you can reuse it? (Obv I refer to boring-to-write non-performance-sensitive code here)
Ya, I think it's great that Python has become a sorts of universal / unified language these days. When I was a student, people used a bunch of different languages to write software, depending on what group they were in, or what the culture there was.
You had Matlab, R, Octave, Java, C++, Lisp, etc. all depending on what type of scientist or engineers you worked with (Statisticians used R/SPSS/etc., Physicists and Electrical Engineers used Matlab or C++, Computer Scientists used Lisp etc., and the list goes on)
At least today, most is written in Python - only the libraries are different.
Also, sharing code is a breeze today. Back then you'd have to download or manually transfer the codebase from other users, check dependencies, install software, check licensing, and what not.
Today you can fire up a Python notebook, and that's it - very nice for smaller stuff. Even larger applications is a easy as pie.
>_Physicists and Electrical Engineers used Matlab or C++_
Trained as an EE and this is true. We used MATLAB/Simulink for signal processing and control systems. Octave, too (because our sensor technology teacher was more comfortable with that). x86 and Microship Assembly, C, and Pascal. Used LTSpice and NI Multisim for circuit design and simulation. Cadsoft EAGLE for printed circuit board (PCB) work. Played a bit with LabVIEW.
However, I started coding as a child with BASIC/x86 Assembly/C/Pascal/VB/Delphi. Doing mostly Python for the past several years. I suppose one could find reasons to "love" or "hate" anything, but I always found that nuance keeps sanity.
> I think one thing programmers underestimate is how many scientists and academics are using Python. It's not just ML, it's pretty much every kind of scientist and tons of non-science academics too. The fact that it's a great "casual" or part-time language is huge for these people. They aren't full time (or even self-identifying) programmers, but their programs are non-trivial. The readability, portability, tooling, and massive package ecosystem is huge for them.
I've found the reliability and tooling really unsatisfactory, at least compared to Haskell.
The libraries are there, though, and familiarity is in the departments (psychology (maybe more R?), physics, &c.)
Oh yeah. Lots of scientists just want to turn their text file of data points into a graph, and Python is great for that.
Some people just stick to the basic stuff but there's plenty of other, more complicated programs that are developed by academics/students. Guess what they're going to use? The language they first used to make that pretty graph.
We need to start ranking languages based on what people use them for.
Python is certainly #1 in ML, for example, but far from #1 in web.
I had a "which stack" argument with a client recently for a web server processing tons of data. He wanted to use Node (which I hate, but also use when performance doesn't matter because TypeScript is so good).
I showed him that the Node library ecosystem is actually pretty weak (many libraries, but most abandoned, single-contributor, and/or low-quality). Of course, browser libs for JS are amazing.
My point is just that mixing all these things together in popularity rankings is pretty meaningless.
I went from Rails to a Node-based stack with a job change recently and the backend libs we're using feel comparitively like pushing boulders uphill, which I didn't expect. (There may be more/better options than what we're using of course.)
Agreed. I find with JS I have to write a lot of code for features that could be handled with a gem in the Rails world. I miss gems like devise that had large amounts of common functionality that there is zero value in me re-implementing myself.
What libs are you using? I'm using Typescript with Tsoa and Objection and it's pretty good.
You are probably using TypeScript. Try vanilla Node/JS if you want a faster Rails.
What does typescript have to do with speed?
Static typing can be a bitch to get right. There are quite a few weird cases where it would be natural to write without types, but tricky once you introduce static typing and want to keep pleasing the TypeScript type checker, usually involving generics.
> Static typing can be a bitch to get right.
That goes away with experience. Typescript can be particularly bad because it has in incredibly complicated type system (to accommodate all the wacky things you can do in JS), but after you get "used to it", the puzzled pauses become fewer and fewer. Eventually it's no inhibition whatsoever, and actually makes you faster because of IDE code completion, refactoring, and bugs caught by the IDE/compiler.
This is that "blub" thing people talk about.
Static typing is an enormous time-saver. I would never, never, never use JS without it again.
Types don’t affect runtime performance unless you’re changing the algorithm. Maybe you’re arguing that correctly annotating certain algorithms is harder than switching to a slower but easier-to-annotate algorithm, but I’m very skeptical of this, especially since (in my experience) well-typed programs seem to be more efficient (easier for the compiler to optimize types that don’t change at runtime than those that do).
I wasn't talking about runtime performance at all. I was talking about coding speed. (I don't think the commenters before me were talking about runtime performance either? Did I misunderstand?)
I interpreted this as a comment about runtime performance:
> Try vanilla Node/JS if you want a faster Rails.
Understanding now that you meant development velocity (or whichever term), I contest that static typing is in impediment in general, but rather that it's an impediment for people who aren't used to it. Especially for those who don't care if their code is correct or if it only appears to be correct superficially, although someone will inevitably (and authoritatively) drag out some decades-old study which finds no significant benefit to static typing when comparing Python/Lisp to C++89/Java or something.
Dynamically typed, vanilla JS always seems like the faster option at first. Then your team grows, new developers inherit the project, and your codebase needs to be worked on at scale. This is when vanilla JS falls apart and becomes incredibly annoying to maintain, and it's why most of the big tech companies out there do not use vanilla Node JS for large projects.
Types are like living documentation - a contract between devs that describes and enforces specific behavior. Just because you know what properties an object contains now doesn't mean you'll remember it in a month, or that the dev who inherits your work will have an easy time figuring it out. I feel way faster when I code using types, because the type system tells me exactly what I'm working with at a given time. This ends up preventing lots of bugs and makes debugging errors that do happen way easier, in my experience.
So TL;DR types seem slower at first but the dividends are paid in full, and then some, when you have to maintain a project among many devs or teams of devs.
EDIT: not to mention that IDE autocomplete features often seem like a superpower when working with static types. I'd recommend Typescript for that reason alone, honestly.
Relying on the type system for code insight and bug catching means one have not carefully studied the code, which is what causes most bugs. If the code base is too large for anyone to digest, then break up responsibility.
This question doesn't make sense. The comment is vanilla JS as "a faster Rails", not "vanilla JS is faster than TypeScript"...
https://spectrum.ieee.org/static/interactive-the-top-program...
[Access to this page may require you to create an account.]
For years, IEEE has ranked the top 55 languages based on five use cases typical of IEEE members: overall, web, enterprise, mobile, and embedded, and ranked three ways: trending overall, in demand for jobs, and used by open source.
The 2020 results, ranked overall:
1 Python 100.0
2 Java 95.3
3 C 94.6
4 C++ 87.0
5 JavaScript 79.5
6 R 78.6
7 Arduino 73.2
8 Go 73.1
9 Swift 70.5
10 Matlab 68.4
11 Ruby 66.8
12 Dart 65.6
13 SQL 64.6
14 PHP 63.8
15 Assembly 63.7
16 Scala 63.5
17 HTML 61.4
18 Kotlin 57.8
19 Julia 56.0
20 Rust 55.6
21 Shell 52.0
22 Processing 49.2
23 C# 48.1
24 SAS 45.2
25 Fortran 43.0
26 Cuda 41.0
27 Visual Basic 40.3
28 Objective-C 38.9
29 Delphi 38.6
30 Perl 38.2
31 Verilog 37.6
32 VHDL 36.7
33 LabView 36.7
34 Elixir 35.8
35 F# 34.7
36 Prolog 34.6
37 Lua 34.4
38 Lisp 33.0
39 Ada 32.8
40 Apache Groovy 32.0
41 Scheme 31.4
42 Haskell 30.8
43 Cobol 30.4
44 Clojure 29.8
45 ABAP 29.5
46 D 27.7
47 Forth 23.7
48 Ocaml 23.7
49 TCL 22.1
50 LadderLogic 19.5
51 Erlang 18.3
52 Eiffel 16.5
53 CoffeeScript 15.9
54 J 14.3
55 Racket 0.0
Just curious, what other server side languages do you feel have a better library ecosystem than Node/JS?
Depends what you mean by "library ecosystem". If it's raw number of packages then npm is number one. On direct opposite side of spectrum you can find e.g. Elixir/Phoenix
- relatively small number of packages in registry but even built-in stuff covers 99% of what you want - and makes this effortless for you.
E.g. after single `mix phx.new YourAppName`
You get
* Pre-confugured prod/dev/test environments
* Preconfigured webpack for the environments above
* DB migrations
* telemetry/metrics with live dashboard
etc
P.S. encrypted signed cookies, changesets (validations for db models or arbitrary forms), clustering, websocket stuff, "liveview" serverside rendering ...
I've heard great things from Elixer/Phoenix fans, so I'm sure it's a great framework, but I'd argue you get a pretty comparable set of features with some of the most popular project templates for the bigger, older frameworks like Django or even Rails.
That said, I've seen that "liveview" thing demoed and that is definitely pretty slick. Off topic but it's very similar to a package I implemented in Django like 6 years ago or so (and happened to name it the exact same thing, too) --- sadly was for work in a proprietary setting, although I'm off-and-on working on remaking it now for a different project where I can open source it.
JVM based languages? Also I like the .NET ecosystem as it comes with batteries included and is quick to develop with.
I'm finding that JVM is really starting to look threadbare as a web development language.
I haven't tried Spring Boot, so I can't speak to that, but gather that a lot of people like it. Otherwise, though, it's kind of a mess of too @#$% many options and no clear winner aside from the legacy stuff that nobody actually enjoys working with on the flagship language, an alternative language that was initially promising for web development but ultimately collapsed under the weight of its own creeping featuritis, an alternative language that collapsed under the weight of efforts to turn it into Jonathan Livingston Programming Language, an alternative language that collapsed under the weight of a cult of personality, and an alternative language that looks poised to collapse under the weight of Oracle v. Google.
Everyone has different ideas about what matters; I care about continuity and stability. I cannot do a monthly dance of 'library blah is no longer maintained, please try library vlah which, by the way, is completely incompatible API wise!'. I need stuff that runs fine in a decade and where the maintainers respect backwards compatibility. I have not been let down by the past decades of JVM (and PHP but I never used many libraries there, so not a good example) and last decade of .NET in that regard. I carefully make sure i know my dependencies (read the code, check maintainers; ecosystems that have millions of tiny libraries cannot work for me; I like my sleep). Things simply work and they work, without much issues, after major upgrades of libraries and runtimes throughout the years. My business goals clash with fast moving things; my clients don't want them and I don't want to maintain them. And yet my software runs under the hood of 'hip' companies; they build fast moving stuff on top of it; I like it that way.
I think we may be speaking to the same goals. My worry about the JVM is that, culturally speaking, the community has been losing interest in the old stable stuff (Which I get, it all _looks_ like 20 years ago.), and so the Java ecosystem seems to be eagerly headed in a more "move fast and break things" direction.
Well... Compared to others it is still very conservative. And backward compatibility is taken into account for many projects.
I too find it annoying that everything feels like early 00s as it prevents uptake; I think most can be fixed by creating a new, modern website for the project, adding examples how to interact with the system with modern tools and a 'try it out in your browser'. Many Apache projects, for instance, are great and very robust in the current time, however, when you go there to do research on what to use for a new project, the examples start with XML files and then Java classes. No matter what I think of node, examples these days must include a little few-liner with node, python, go, curl, java, clojure, ... . Most people like the idea of java -jar somethinggreat.jar, but opening IntelliJ, adding all kinds of arcane (when you are used to Go, Node, etc) artifacts and then writing actual Java code which you did not want to do in the first place, really hurts adoption imho. And made many projects not even appear on anyone's radar.
Like (anecdotally) picking mongo for a (trading) case geode was basically made for, but mongo was hip, not 'old apache' and not 'old java'.
JVM is still doing well in terms of interest.
The declining interest in Java is offset by Kotlin, Scala, and Clojure.
Kotlin is ranked _really_ low. Much lower than I would expect given that it is almost an alternate Android language.
Did the Dart guys somehow gain massive momentum? Or is Electron or React eating this up?
For my part, as someone who's just starting to look into cross platform development again after a couple decades of not caring, I have been 100% focused on Cordova and React Native. JavaScript on the front end seems like the only viable option, so I'm going with it despite my strong distaste for JavaScript.
I dismissed everything else pretty quickly for being too limited in what platforms I could target. Kotlin's out for being a poor choice for iOS. Dart/Flutter is out for not yet having fully baked desktop support.
Python.
The performance lacks somewhat and the debugging tools are not as mature as JVM's. But the libraries, frameworks, and overall development productivity are great.
Some of the newer ASGI frameworks are actually pretty good performance-wise. I haven't touched Node.js in years and recently needed a backend for an iOS project with a lot of geoJSON processing, used FastAPI and found it excellent. While Python is far from perfect it opens you up to a diverse ecosystem from AI/ML/Geo/science communities.
Java and .NET.
Not only better library ecosystem, more performance and tooling.
PHP has a solid library ecosystem, especially around Laravel, which is what I know best.
I'm not sure I agree with that. Lots of stale, abandoned libraries in PEAR/packagist as well.
Most libraries exposed in some sort of package repository in PHP land are pretty bad. In terms of quality, but also in terms of documentation (frequently not there, partially there or wrong). It's not even rare to find libraries wrapping things with only half of the expected functionality implemented.
To say nothing of trying to get patches into the core. Be prepared to wait months for any response at all, every time getting the feedback that there is some new minor problem (or even worse, a "this needs to be talked about" without any followup).
It's not a healthy ecosystem. For that you require high-quality core libs, a strong developer ecosystem (that doesn't mean a lot of devs, it means good ones) and a culture of third-party libraries being few but high-quality. Anything else is going to end up pissing you off, guaranteed.
Java ? the amount of crap that is available for Java due to its 25 years alive beats the amount of crap that is available for NodeJS.
I program in Node/JS currently and programmed in Java about 10 years ago. But my preferred language will always be Ruby. I will never understand why "the community" (ML, scripting etc) preferred Python to Ruby.
JVM, Python, and .NET all have better server-side ecosystems (which I know from actual experience on multiple projects).
The original project I was discussing ended up going with Kotlin on the server, and everyone (including devs) are super happy with it.
Ruby / Ruby on Rails as well
I think basically every majorly used general programming language has better libraries than Node - which becomes problematic if your application needs to do something that you would normally expect to get by using a library - but the thing that Node has which in my estimation makes it better than almost every other language is I don't have to context switch when changing languages in between backend and frontend.
I'm actually surprised by this sentiment whenever I see it. Node still hasn't caught up in coverage. It feels like the Node community is really proud of their number of packages, which, given the likely median size of each package, is a poor proxy for actual stability and completeness.
Java
async python
A totally valid point.
Do keep in mind that one of the reasons people choose Python though, is exactly _because_ it is used in a few different domains. E.g. I'd rather my server-side language be the same as my video-processing language be the same as my data-science language, it makes things easier.
What makes you so certain it is far from #1 in web? Django is massively popular as is flask.
No estimate is perfect, but PHP is far and away #1 because of WordPress.
Here are some stats that you could and should take with a grain of salt:
https://w3techs.com/technologies/overview/programming_langua...
My anecdata also says neither Django nor Flask are very populate. I have yet to encounter either one already in use by a client. Rails is also rare. I think scrappy startup frameworks are a different population than what most devs (who work at big firms) use.
Interesting how experiences can be different. I have worked in F100 companies and startups and saw a lot of Django/Flask :), Bank of America was hiring flask devs like crazy back in the UK.
One of the first things I made in python, over a decade ago, was a web scraper hooked up with an OCR-software suite to harvest a lot of data on the web. I imagine that's sort of a proto-version of some of the kinds of machine vision tasks that are done with it today.
This is just TIOBE, nothing more. If one wants other markers, it's possible to use different benchmarks/indexes. For example, I like this one:
https://www.techempower.com/benchmarks/
i think flask and django are quite popular.
Go has the best std lib of all.
Python is certainly popular, but the TIOBE Index ranking is not credible.
Quote: "The ratings are based on the number of skilled engineers world-wide, courses and third party vendors. "
According to this index, "Logo", beats out Rust, Scala, Kotlin, and Cobol.
I cannot remind myself when I last time saw a course, book or even any source code covering Logo.
This must be some very poorly vetted, automatically collected data that tripped over something provoking the ranking.
They rank C number 1. There is no way that reflects reality.
JavaScript is easily number one for reasons that will surprise nobody. On the WakaTime public leaderboards it dominates to such an extent that I think programmers spend more time writing JavaScript than all the other languages combined. I don't know if that generalizes to the broader population, but it gives one an idea how ubiquitous it is.
I can't actually say what the popularity of C/C++ is.
But I don't know what you do for a living but in general Hackernews tends to have webdev tunnel vision. Embedded, operating system development, browser development, OS, driver dev, games, telecoms, compilers, lots of IoT, systems development generally, all of this is in C/C++, and will be for the forseeable future.
That is a lot of lines of code, but mostly invisible to people whose role in the industry is "full stack development" type jobs.
I've worked (not just played with) almost twenty languages in a career about the same length in years.
I'm well aware of the things C and C++ are used for. I'm currently mostly working in Rust at that moment to make use of libraries in C++. But I think web development easily swamps that kind of work in terms of number of developers.
> But I think web development easily swamps that kind of work in terms of number of developers.
does it though. IBM alone, a service company, has 380000 employees. And there are _tons_ of service companies like that - at least in my country they are mostly doing Java and the like.
I do most of my work in C/C++. There's a ton of it in every microwave oven, washing machine, car, telephone, device driver, etc.
Likewise, my day job has been C++ for the last decade. And it's not just in devices or games, or in OSs. Before I worked in hardware devices I worked in video pipeline stuff in Google Fiber. And before that, I worked on the ad serving stack at Google and another company before that. All in C++. A good portion of servers @ Google are in C++.
Now, if you strictly interpret the "C" in this popularity nonsense as "C", then sure. Pure ANSI C has a fairly restrictive use environment. But "C/C++" is a much much larger ecosystem.
Also get a laugh out of the people here calling C "simple". Ho boy.
Maybe I'm crazy, but recently I had to dive into ANSI C in a big way for some computer music projects (many old computer music code bases are pure C) and I'm really liking it. It's like you have a handful of really damn sharp tools and you can do amazing things with only those if you keep in mind you gotta keep your fingers clear of the blades.
Was going to say the same thing. Considering total amount of running code, yeah C might be #1 but there's no way it's the most "popular".
> They rank C number 1. There is no way that reflects reality
Get a thousand random developers and ask them to list all the languages they know well. What do you think will be the most common?
C seems quite realistic given what they measure.
That C is supposed to be the most popular language makes the whole ranking questionable.
The stackoverflow ranking[1] corresponds much more to my own experience in real life.
[1]
https://insights.stackoverflow.com/survey/2020#most-popular-...
I'd argue that StackOverflow shows what developers have the most questions about. This means that ranking is influenced by the amount of experience people have in the programming field, and leads to languages people commonly use as their first languages showing higher.
Besides that it's also influenced by how good the language is at explaining to the user what is going wrong. Clear error messages mean the user uses StackOverflow less, while arcane errors would mean they need to look for longer for the solution for their issue.
The stackoverflow ranking we are talking about is the result of a survey. It's not influenced by amount of experience. Experienced developers don't get greater or lower weighting in that survey.
True, however, the primary problem with the SO survey is the participants are self-selected [1], not randomly sampled. (It's explained in the survey's methodology).
What this means is we have no idea if the sample of programmers is representative of the worldwide programmer population.
I'd argue that C# and other Microsoft languages are vastly under-represented in the SO survey, but we have no way of knowing if that's true.
[1]
https://en.wikipedia.org/wiki/Self-selection_bias
C is the most popular language for embedded systems development by a huge margin.
StackOverflow is also a place for asking questions. C is a relatively simple language and one which has been around for longer than most developers have been alive. There may just not be much left to ask about it.
Just about every larger company has an IT department where people write business or web applications. But not every larger company has people programming embedded systems. The TIOBE index simply does not reflect reality.
Every car, every industrial machine, every rocket, plane and tractor as well as the entire stack that your web applications run on in the cloud, the operating system, the infrastructure have significant parts written in C or C++. I think you need to take into account that the world of software extends beyond web applications and white collar services.
I think what they're saying is that applications are an order of magnitude more numerous than infrastructure software. Sounds plausible to me.
Is it ?
We sure are using more apps than ever. But I think we are also using more "smart" things than ever, and not just in the buzzword sense.
A ton of everyday object now have chips, from toothbrushes, thermometers, door bells, car keys, fire alarms, lights, even charging cables.
Considering drivers are coded for every variations of these appliances and chips, it look to me to be a tremendous amoubt of code written all over the world. Especially as writing code is cheaper than adapting hardware, even different uses of the same chip could get different drivers.
Well, if you take away HTML/CSS and SQL, because most people would not call them programming languages, Python still comes in second, right above Java. So it's alright.
How could they even include HTML/CSS? A huge majority of people who are using Python, Java, .net, and JavaScript are going to answer yes to those.
SQL is justified though. I’ve met quite a few BAs or DBAs who can do magic in a sql developer suite but don’t have a programming bone in their body.
IMO, this is one of those theory vs practice things. CSS is turing-complete, but that’s really not how it’s used. So, excluding CSS really should also exclude SQL, though not TSQL.
I'm mostly a C++ and Python programmer by a large margin, but I only ever visit Stackoverflow for Javascript questions or Linux distribution bugs/misfeatures.
This only represents the sad state of Javascript documentation, I think.
You been to MDN? their JavaScript Documentation is impeccable? What are you referring to? There are tons of JS resources available? Or are you referring to 3p libraries which vary in terms of their documentation and can be contributed by anyone?
> You been to MDN?
Of course.
> ...their JavaScript Documentation is impeccable?
Well, no. MDN doesn't answer the questions everyone has: "Which browser does this work on?" "Is it okay to use this new feature?" "Why does this buggy thing do this to me?" "Is it really true that you can't do <simple thing> in CSS"?
Etc. Web standards really aren't. (And I'm not even touching the third-party library churn problem here.)
I mostly agree. I tend to ask and follow peers advice rather than what is immediately available on the web.
On the other hand the stackoverflow survey might show that C is so simple that programmers using it don't need stackoverflow :)
C is simple on the surface but it gives you enough rope to shoot yourself in the foot that even after 6 years of using it I don't dare call it simple knowing my war scars.
Sure, but you probably don't go to stackoverflow to help you debug it. Stackoverflow is more for "how do I do X in Y" rather than "where is the error in my million line codebase"
I think there is a difference between simple and easy. I would say C is simple but still hard ;) I'm not a native english speaker so take this with a grain of salt.
I was a big stackoverflow user and then completely stopped using it when I switched to full time embedded C. Any APIs I was using or problems I hit were too niche.
C is simple? I don't find it simple compared to Python or other web langs.
It's just a complete joke.
https://www.tiobe.com/tiobe-index/visual-basic-dotnet/
why?
> based on the number of skilled engineers world-wide
So I guess it just doesn't count JS developers as skilled engineers?
Logo is still very commonly used in early educational settings - maybe not typically for engineering applications, but I'm confident it has far more people using it than the other languages you list.
I recently had to use logo when migrating a model to another platform someone originally wrote in a flavor of logo. It's a mess.
The Logo code was a mess, or the Logo language is a mess?
I have a hard time believing that Logo should even be used by a fraction of the people using for example Kotlin, which is a very popular language for Android development ...
And yet... there are currently more elementary school children in California than there are Android apps in the world.
(The parent link has a quote from TIOBE CEO that illustrates they are not considering only professional software engineer use)
The past month I have used Python again after a few years and I have to say I didn't quite like the experience as much as before. On one hand I also used vanilla Python instead of something easier to use like Django but the amount of preventable run-time errors and the lack of IDE autocomplete support at various points was kind of off-putting after coming back from very strictly typed languages like Swift, Kotlin or F#.
It also didn't have the simplicity that Phoenix / Elixir still has, for example when you Google how to solve problems you have Python 2 versus 3, the old way to handle requests which is limited and the new way to handle requests which isn't as documented, yes you have type hints and named arguments but they're not everywhere yet and so on.
So I don't think I will circle back to use Python again unless there's some kind of reason I can't easily go around it. There are still some very good ideas in there (I like the braceless approach to code hierarchy and a standard on code formatting in general) and it's definitely not as dumb as JavaScript but it has accumulated too much cruft.
dataclasses are the best recent addition to Python, IMO, maybe tied with ordered dicts.
Python definitely doesn't have conceptual simplicity, because there are many competing concepts in the language, and lack of focus on uniformity. (Example - enums are implemented with a metaclass and dataclasses with a class decorator - arguably the latter is the wrong choice, because it is already evident it limits what kind of features can be added to dataclasses).
Another example just because it makes me irate - `typing.NamedTuple` has been added as an "alternative" to `collections.namedtuple`. Same name, new capitalization, slightly different features. Add to that, a collection type that's inside the module `typing` which is supposed to be for.. type annotations, not actual implementations.
Author of dataclasses here.
> (Example - enums are implemented with a metaclass and dataclasses with a class decorator - arguably the latter is the wrong choice, because it is already evident it limits what kind of features can be added to dataclasses).
I'm curious about what features can't be added to dataclasses because it's not using a metaclass.
If anything, I think that not using a metaclasses is a good thing: it allows you to use @dataclass with classes that do require a metaclass. That metaclass would likely be incompatible with the metaclass that such a hypothetical @dataclass implementation would use.
Thanks for not using metaclasses for dataclasses. I’m looking forward to all libraries replacing metaclasses. For instance, in Django, I think all their metaclasses can be replaced by __init__subclass__. The main issue I have with them is how difficult it can be to compose a new abstract class using upstream classes with them.
I love dataclasses, incredibly simple to use but so useful. My only gripe is no first-class support for "__slots__".
I have a decorator to add support for slots:
https://github.com/ericvsmith/dataclasses/blob/master/datacl...
(although I'll admit I haven't tested it in a while).
The reason it's not an option by default is because it would be the one case where the decorator would have to return a new class, and I didn't want to do that on the first version. As it is, @dataclass just modifies an existing class. I might bite the bullet add a slots option, and we've had discussions on adding a language feature to automatically add slots to a class, based on annotations and maybe some other magic. If we did that, we wouldn't need to return a new class. But it's not a front-burner task for me.
I really appreciate it! Only issue I see is that it won't support inherited dataclasses where the base class already defines __slots__, but I should be able to hack together support.
I believe this is the example where dataclasses could have supported __slots__ better with a metaclass?
Thanks for working on dataclasses :) - as I said, it's my #1 new feature in Python.
Maybe that would have worked better with a metaclass.
But still, I think not using a metaclass is the more flexible design. Maybe I'll add the "add_slots" behavior into the @dataclass decorator in 3.10, even though it would need to return a new class. At least it could be well documented, and I doubt it's a concern for most people.
> Thanks for working on dataclasses :) - as I said, it's my #1 new feature in Python.
I'm offended! I also wrote f-strings, but maybe that's not considered new anymore. In any event, you're welcome!
Maybe the __slots__ discussion is not really that important - it's just convenience, and the regular old method of adding slots still works. So it's a minor thing.
I have diligently converted to f-strings but it doesn't give me new expressibility like I feel dataclasses do, just convenience :)
fstrings and datclasses changed my life. thank you :)
FYI if you haven't heard of it, attrs (
https://www.attrs.org/en/stable/
) is a package very similar to dataclasses that does support aromatically adding __slots__. I believe dataclasses was bases on it.
Yes, attrs is the spiritual parent of dataclasses, including the decision to not use metaclasses. Thanks for mentioning this: I try to always give credit to Hynek and attrs.
*based. Also @ericvsmith please correct me if I'm wrong - don't remember where I read that.
No, (optional) static typing is the best recent addition to Python. It allows IDEs to catch a large variety of errors. The static type system is part of the language and has multiple implementations. It's Hindley-Milner, making it two-way, like Haskell's type system.
There is no "No", you can just add your opinion about your favourite feature instead.
Wait what? Are you saying Python has a Hindley-Milner type system?
What's it called? Google didn't turn up anything.
It's called PEP-484, and it's part of the language
https://www.python.org/dev/peps/pep-0484/
Pyre and PyCharm are third-party implementations of PEP-484.
Python doesn't have it but I think they are talking about mypy, options typechecker tool that has type inference.
Pycharm autocomplete is very good - I don’t know if that is what you mean by autocomplete support
Requests is one of the modules that had more change on 2-3, but
being basically the most popular language (for good reason), it’s pretty easy to find documentation for either version.
I imagine a lot more than for Phoenix/elixir whatever that is.
I think you’re being difficult for the sake of being difficult. I am not a leet programmer and that is why I use python- it does everything for you very well and no more. If you want more “cruft” pip install cruft
You might be surprised to see the quality of the docs in the Elixir world:
https://hexdocs.pm/phoenix/Phoenix.html
That looks … like many Python projects? It's slightly prettier than the default Sphinx theme but I'm not seeing anything which wasn't common a decade or two back.
IMO part of the quality here is these docs are generated from comments in the code. All you have to do is document your functions and modules properly, run a command, and you have an entire set of docs.
It reminds me a lot of JavaDocs, although I never used them extensively. I'm also not very aware of how documentation looks in the Python world.
Another interesting thing is the code examples. You write them in the comments as "Doc tests", and they are run and tested like normal tests. This means that all the code snippets are up to date with the latest version of the project (as long as you do test!).
The best part, also imo of course, is that it all works out of the box.
That’s how it works in Python, too. You add text descriptions (docstrings), automatic documentation generated from function/class definitions (comments can add prose), and you can run doctests to test things like examples.
See e.g.
https://www.sphinx-doc.org/en/master/index.html
and
https://docs.python.org/3/library/doctest.html
Very nice, I'm glad to see it elsewhere. Maybe Elixir even took inspiration (or it's just a plain good idea). I'm really surprised that I had never heard of or encountered it, maybe it's just an accepted part of the ecosystem so no one talks about it and/or I assumed everything was written manually. Also haven't used Python professionally which doesn't help.
Yeah, I don't care who came up with it first as much as it spreading widely. We waste far too much time on bad documentation as a field.
> I think you’re being difficult for the sake of being difficult.
Why? I was just trying to get something done that I've done in a variety of different languages already so there's a perfect comparison.
> I imagine a lot more than for Phoenix/elixir whatever that is.
I don't know how you manage to come to Hacker News often enough to find exactly my post to answer to but at the same time managed to have never heard about Phoenix/Elixir. What's the point of coming here if you're not interested in staying current in your field?
I moved/was pushed over to Python around a year ago, after writing C# for 15 years.
My Python setup is as follows:
* vscode
* pylance
* black formatter
* flake8
* python typing annotations used as much as possible
This seems to work very well with regards to autocomplete and preventing run-time errors.
One thing I do miss is proper refactoring support.
> One thing I do miss is proper refactoring support.
You should try PyCharm then. That, and their debugger is orders of magnitudes better than VSCode.
Seconded. I had an IDE I was used to and loved and the first time I pair-programmed with someone using PyCharm I threw it out and switched. Doesn't mean it will be the same for you, but it's miles ahead of everything else I have used in 15 years for Python. FWIW, I like VSCode just fine for everything else, PyCharm is just so handy.
I did try PyCharm a few times, but it seemed too complicated - and this was after using Visual Studio for 15 years!
The debugging support in VS Code is fairly good, so it's just the refactoring I'm missing.
I'll give PyCharm another go when I get some spare time.
I don't understand why someone would use Python instead of say using Ruby. I think right now it is a matter of "synergy" given that Pandas, Torch and all those libraries are more tailored for Python, but as a scripting language Ruby is just so much easier, elegant and intuitive [ len(something) vs something.length ]
FWIW, I found Python far more intuitive than Ruby. For the most part, Python works like I expect it to, meaning I can write large swaths of functional code without having to look at documentation, something I was never able to do in Ruby.
> [ len(something) vs something.length ]
The first reads like english, the second doesn't.
> as dumb as JavaScript
Could you elaborate on this?
Much better base packages. Date handling. Type hints. Not an OO fan but it does have real classes. IMHO handles complex applications better.
Though Python can look a bit more clunky than modern JS.
What IDE do you use, that doesn’t have full autocomplete support?
Gotta get the js digs in. Nice.
> The past month I have used Python again after a few years and I have to say I didn't quite like the experience as much as before.
Python is really a Perl replacement. It was meant for the scripting world, not software development - pretty much stuff that was too complex or cumbersome to do in bash.
Your blaming Python because you used a poor IDE and failed to use type annotations?
I'm using PyCharm, if you have anything better than that don't hesitate. But the type annotations are obviously something that also needs to be added to all other code I touch, not just my own.
I’ve used PyCharm from time to time and I find it great but I kind of feel like you need a different approach. Coming from Java it made sense but Python is a different mindset once you get into it. I think you’re far better off treating Python as a highly structured scripting language and using an advanced text editor. I’ve come to believe that these very powerful IDEs are a kind of a window dressing that makes the more serious languages easier to work with. With more flexible languages the language itself is the tooling but it’s not so easy to audit at the end of the day.
Although arguably a more 'difficult' language I sleep much better at night when using Rust, because Python is always in a hurry to get a wrong program running. So then, one has to test like hell. So I feel I'm still ahead of the game.
It’s horses for courses. I’d never use Python for the kind of thing that rust is for. At the same time, I’ve never had occasion to use rust for the stuff I do.
EDIT - just want to add: I'm not au fait with the specifics of Rust but I presume from what I've heard about it that it also supports my position, as an advanced language, that the features you need are in the language so a heavy-weight IDE isn't so necessary.
eclipse + pydev + mypy/pylint integration
I personally find the methodology used in TIOBE very dubious. It frequently has very large differences with other metrics, and often "feels wrong" compared to those other metrics.
It mainly seems to massively overvalue historical significance, the older the language the higher it is. E.g. the current ranking has Perl 5x more popular than Typescript, and Visual Basic 2x as popular than PHP.
I will say that I've been coding for almost 20 years and I think I've only had to touch perl once, so you may be right about that.
It's definitely the case that Python is very popular though. It's everywhere. As others have pointed out, it's become very prevalent in scientific/academic circles. Outside of that, it's very useful for scripting. They teach it in universities as an introductory language. It's in use at every workplace I've been.
It is biased towards (European) industry but
https://stackoverflow.com/jobs
is a good metric on language _demand_.
Since I started to follow it systematically (mid-2019), the trends I perceive are: Java or Scala -> Kotlin, JS -> Typescript, Angular -> Vue. Growth of (non-web) Python, React, Go and even PHP. Besides Kotlin, mobile tech demand seems stalled (Swift, React native, etc). Frontend demand is shrinking.
Like many people here, I've worked on large python deploys for most of my professional career and have, over time, come to see python's downsides more clearly.
I also wonder, thinking back over my experiences and reading others here, if we are mis-ordering how these things come to be. Maybe it's not that python codebases are, for a given line count, worse than other languages. Maybe it's that python allows teams to continue using practices that aren't suited to their current scale longer than other languages. The reason that we all have seen hulking, monstrous, nearly unworkable python code bases is that most other languages would have already collapsed under the mismatch between approach and desired outcome.
I think we often approach software engineering as a puzzle - where you have a set of inputs (a too-large codebase, for instance) and a question of how to a better state. But programming projects are path-dependent creatures. Huge codebases must develop over time - they don't spring out of the heads of developers fully formed. If you were a typed language and your engineers had a lower LOC output per-day, then of course your code base will be smaller. Is that better? You iterate more slowly, but the scale of your code is also more manageable. In my experience, the challenge of large python code bases comes from the context that you need to understand from the surrounding environment: what are all these objects, what are their objects, etc. Typed languages force you to carry around more context, so any given function is easier to read, but you can still write un-navigatable code.
A point I think people miss with Python is they talk a lot about 'Python Software Developers', but I think the growth is coming from areas where people are NOT developers, but are using scripting/automation in other fields. This is a much larger market than software development, and people will work very differently compared to developers.
And they have C as number one? Visual Basic ranked higher than JavaScript? Groovy (!?) higher than Go, Swift and Ruby? R higher than SQL? What a joke the TIOBE index is.
There are a lot of CPUs in embedded applications, far more than the number of PCs and smartphones, I would expect most of the embedded stuff to be written in C.
Do you believe more devs are working on embedded apps than web apps in 2020?
There are frequently weird things happening with the Groovy rankings in the Tiobe index, and I doubt this is accurate. That said, purely as a language (ignoring the ecosystem) it's probably nicer than Python, and I say that as a primarily Python developer.
I am also surprised that JavaScript is ranked so low.
c _is_ number one.
When I worked at Yelp circa 2013, we were an entirely Python shop (for backend). At that time, we had millions of LOC in Python. I love Python to death, but it became very clear that Python was simply a very bad fit for a project as large as Yelp.
I remember trying to write a program to simply understand our dependency graph so we could decouple some of our services (we were just starting our SOA efforts at that time). This is something other languages give you for free.
Our build times were exceptionally slow, as a direct result of Python's dynamic nature. For example, test _discovery_ took over 2 minutes on each test run. This also made it painful to have distributed test runs, as splitting up tests across multiple processes would require the test discovery step to be run in each process, incurring a 2 minute penalty per process.
We also had deployment issues, because of Python's dynamic nature. Some code paths were only exercised during a production deployment, and were not sufficiently tested before deployment. This is despite the fact that we had tens of thousands of tests and sunk many hundreds of person-hours into testing as much as was possible. Despite a very professional engineering culture, and a constantly expanding test suite covering these corner cases, we had deployments fail about once a week for this reason.
I adore Python and still use it extensively most days (mostly its data science/ML libraries). But I now think that large projects would really benefit from static typing and strict compiler checks, as well as just being able to actually compile a binary (which would have solved many of our testing woes). In contrast to some other smart people on this thread, I don't think that even disciplined developer teams should use Python for large projects, at least not in the long-run.
Your description here is often the same argument for microservices over a justification for any specific language.
Definitely migrating to microservices helped alleviate the pressure from some of these problems: easier testability, smaller individual codebases with simpler dependency graphs, faster test and deploy times for each service, etc.
But the language problem remains. I think there are a lot more good language options today for backend services than there were in 2010, and if I were writing new backend code for a big website/service today, I might pick one of those.
Regression to the mean is the most powerful force in nature. Try as you might, you have to use what everyone else is using. Which is also very hard to continuously do in an industry that has ADD.
Darn it, yet another hint that I'm not getting any younger. 20 years of Java experience and no intention of learning a new language... When prompted, I usually snark that you don't ask a professional violin player to "just" switch to viola either.
Yeah, but in fairness they probably could, a lot easier than switching between programming languages
I'm surprised TIOBE is still taken seriously these days. It certainly sort-of correlates with popularity but it's a bullshit measure.
I think they factor in legacy stuff. There are lots of VB jobs in my area from companies desperate to maintain whatever horrid internal apps they built up 20 years ago.
Along the same lines, IoT depends heavily on C and even stepping out of that, pretty much every piece of silicon requires extensive C code to be written around it.
I still wonder about JS though. Billions of websites with JS and the JS gets rewritten every few years (or less during the framework wars period).
This might be a stupid question but anyway...
Is there someone well-versed in a statically typed language who would still prefer python most of the time?
Could you explain why?
For me it's just the opposite. When I program I think about the types first. When I read code I care more about types than variable names. This is in my opinion the way to go even in C.
Also most python programmers I know are engineers and for them it would be a waste of time to learn C++.
I would like to know why an experienced developer might prefer python ...
Sure — for context I’m very comfortable with Haskell, Typescript, Java and Scala, fairly comfortable with Agda, Coq, C and Rust. I don’t prefer
Python (the reasons of which I can elaborate, if you’re interested), but I do prefer to do my programming in Lisp-like dynamic languages —
even when not using meta-programming.
The reason for this is quite simple. A static type-system, by design is meant to invalidate programs which are incorrect and accept the programs
which are correct. All static type-systems which exist invalidate a certain number of programs which are correct — that is to say, you can
write a perfectly valid program, but the type-system can reject it. What we term to be a more _expressive_ type-system, is one which rejects
more programs. The line which one draws in the sand as expressive _enough_ is completely arbitrary and the type-systems we have currently
heavily rely on the user constructing their programs in a certain way, so the type-system can prove certain properties about it.
Here’s an example demonstrating what I mean. Consider a theoretical language which uses _:_ as the type-ascription,
has generics/kinds and all functions are total. A code-snippet taking the head of a list might look like this:
> let num: Option<Number> = head(numberList)
Note that the return type of the head will be _Option<A>_, if the function is total because the list can be empty. Now suppose I have the snippet —
locally I can reason that the type of _num_ should be _Number_ — but I need to prove that to the type-system. In most current type-systems, proving
this requires you restructure your code in a particular way.
> let numberList = prependList(4, previousList)
> let sortedList = sortList(numberList)
> let num = head(sortedList)
What I’m trying to drive at is that static type-systems are a spectrum and at some point you _will_ run into a case which you’ll either
need to use an escape-hatch or restructure your code in a non-trivial way to satisfy the compiler. The boundary of what you determine
to be correct is also completely arbitrary — you can _always_ encode more properties and restrict your code further; how far you
go is ultimately determined by you.
Thanks for the lengthy reply ;)
I'm aware of the fact that every type system will reject some valid programs. C evades this problem by letting you "escape" the type system. Many people criticize this without being aware that the language would be useless without this feature.
I don't think that an expressive type system rejects more programs though. You don't have to escape the type system that often in C++ (compared to C).
Nonetheless I probably should look up some design patterns you can do in lisp, to get a feel for what I'm missing
When I program I think of business problems to be solved. Types barely register. Yes, on a big project with lots of developers types become more of a priority. At that point I let an IDE worry about them.
This is the reason for me as well. Many problems map very easily and readably to simple lists and dictionaries.
The easiest example is web applications, especially internal ones, which are essentially shuffling JSON messages around from different SaaS and internal APIs. Adding (and testing!) a different ApiResultSerializer in between each of these points can become a huge headache when everything boils down to JSON string or number in your result anyway.
Though I do use C / C++ (recently some Rust) where business problems are not the priority, and I can focus on technical efficiency / correctness.
Python has long been a top-loved programming language, as has Java
Um... nobody _loves_ Java. We use it, but we don't love it.
I used to be turned off by the boilerplate and verbose nature of the language. Over time, having all of this information out in the open removes a lot of guesswork based on tacit knowledge. I also really appreciate its static typing even though that can be limiting.
also, modern java is not really like the ol' days of 1.3 and 1.4. A lot of convenience features since jdk8 -- and particularly the newest versions -- add quality of life improvements that make the language a lot more enjoyable to write.
I think there's a lot to be said for how it's changed, also. The language has experienced improvements, but the community has also matured - a lot of the really awful frameworks and libraries have been replaced by much saner and simpler options.
You can still run into something old and enterprisey, but Java is much less baroque than it used to be for getting things done.
i love it
You don't know any other languages.
Not gp, but I know plenty of other languages and I still love it.
There's nothing to be hated about any programming language ever. People confuse "I don't understand it" with "I hate it".
I've been working with Java professionally since 1999. I understand it as well as anybody, better than most. I don't hate it, but I don't love it either.
Citing tiobe.com:
"The index can be used to check whether your programming skills are still up to date or to make a strategic decision about what programming language should be adopted when starting to build a new software system."
This and the fact that C is number one suggest something very wrong. While C really has a place in my heart, you should have very good reasons to use it when "starting to build a new software system".
> you should have very good reasons to use it when "starting to build a new software system".
And three of those very good reasons are:
1) you're writing a library and you'd like to make it usable from a number of other languages as well, the most popular of which are at least partially implemented in C or C++ themselves to some degree, eg. Java, Python, Perl, Ruby, Haskell etc. Writing said library in C (in particular) makes the library much more accessible from those languages.
2) the platform you're writing for is partially if not largely written in C or C++, the most popular of which are Windows, MacOS, Linux etc. So the toolchain/runtimes used to build the systems are already supported, extremely mature and well tested.
3) the platform you're writing for is especially space constrained (eg. embedded Linux), also fits the criteria for (2) above, and where performance may also be a factor due to reduced CPU/bus speed, availability of RAM and storage. For every interpreter or JIT compiler you want to add to those kind of systems for a higher level language, you can normally fit a larger number of C or C++ programs to implement your device's required functionality.
There is far more years of effort required yet to overcome the inertia of the above than the average HN webdev would realise.
Now that is some epic movement right there. C, Java have been up there in front for decades which I've occasionally explained to nonbelievers by sharing the TIOBE index. The winds of change are blowin'.
By adding stricter typing and a bunch of middleware I can easily imagine a Python EE one day but let's hope that doesn't happen. Duck typing, a taste of functional paradigm, non-verbosity is what makes Python special IMO.
I wonder if Go, Rust might make it to top C one day.
I love Python but I have been hearing this for years. Its programming newbies attempting to gain new skills and searching stackoverflow 100s of times like I used to. Also, I search for jobs online...Python is not the backend tool for the vast majority of companies. Like the creator of Pandas' (Python library) says ....Python is the SECOND best tool for everything.
I might disagree with how "popularity" is computed here, but as long as they have been consistent, the trend is probably real.
IME Java still has a very real stranglehold on business logic code, especially in bigger companies.
Golang and Rust are eating away at it, but more for infrastructure related projects rather than higher-level business logic.
"In the past, most programming activities were performed by software engineers. But programming skills are needed everywhere nowadays and there is a lack of good software developers,"
This is an important distinction and often missed from programming language discussions.
Engineering languages like C++, Java, Rust focus on being able to create robust APIs, handle incredible complexity, deal with dependencies, etc. Hacker languages like Python, PHP, erlang, and Go focus on an approachable language, libraries that tackle problem domains, etc.
There is obviously a lot of overlap. But I think it's good to keep in mind that engineering and hacking don't always align.
Python is an excellent teaching language. It's simple to install. You follow a tutorial by typing something at the REPL and it gives you results. Learning stuff like importing modules, etc. happens as your coding ambitions increase.
Something else to consider is the Python shell at python.org/shell. Students often have Chromebooks. I'm about to mentor second year high school students who are interested in CS. Several of them have Chromebooks only. I can recommend Python and point them to the shell.
I would think Python got a good bump through Hour of Code and other programs to introduce people to coding.
Python is a worthy successor to Java.
people who like top complain about types are invited to run mvn clean install for each change instead of using type hints. I continue to go through that each day. it pays better. Java is the new cobol, hopefully.
Still, with C bring #1, one has top wonder about the quality of the index. Both Python and Java are much more common in the business.
"Popular search engines such as Google, Bing, Yahoo!, Wikipedia, Amazon, YouTube and Baidu are used to calculate the ratings"
These ratings seem pretty meaningless. Visual Basic is #6, tell you pretty much all you need to know.
I'm somewhat tired of claims based on the "scientific" TIOBE index!
Although such rankings are mostly good for entertainment, I'd whish there were a better, more solid index. Redmonk is better, but a bit too limited in its data source (Stackoverlfow and GitHub).
Python for toy projects, scripting, ML and Java for real scalable applications is the way to go IMO.
I got back into python recently and one thing that really confused me: there's no good way to manage packages, apparently? Meaning: if someone clones my repo, there's no single, correct way for me to have specified the dependencies in a machine-readable format, so they can run the equivalent of "npm install", and get things working. Is that really true?
Use poetry.
No, look up "pip"
pip install -r requirements.txt
Makes sense due to the proliferation of new JVM languages.
Thats says lot of amateurs are doing programming today, who have no idea what programming is.
Of the top 10, only Java has experienced a year on year drop.
So, at least some of the gains are at the expense of Java.
For C and C++, you're probably not going to reach for Python as a replacement. It's not like in the 90s when these were considered general purpose. If you're using them these days, it's more likely due to needing to use them.
Java is still used for a lot of other things where a language like Python would be fine and likely more productive. Also, I have a feeling that the large growth in go from #20 to #13 likely also came at the expense of Java.
That, and the pie is growing. Python is taking a larger share of the larger pie.
On my area of work only Java and .NET get to play, and in what concerns UNIX like platforms most customers only care about Java.
The only one that is growing, although I rather not, is Go, as we now start to jump into the same container bandwagon as everyone else. And even there it isn't taking anything away from our Java/.NET setups, rather complements them.
If you need more speed than Python can offer, but still care about developer productivity, Go is pretty great.
Python is ideal as BASIC replacement and scripting language, for being a glue language for C and C++ libraries, almost any language can do that.
As for Go, when they adopt generics, until then I rather spend my productivity in Java and .NET land, and only deal with it in the context of containers eco-system.
What are good reasons to choose Go? If I need a language for non-scripting tasks, I would choose Ada, which is very easy to read, has a nice type system and generics, performance, and you could statically compile everything if you wanted.
I don't know anything about Ada, so perhaps that is a better choice!
But for my personal experience Go's concurrency features, type system, and simplicity is pretty great for getting up and going quickly with performant code.
This sounds very similar to the shop I'm in. Go isn't replacing any of our legacy Java stuff, but we have new tooling to supplement the legacy stuff built in Go.
If it's the second, what's the first? PHP? :-)
I don't know why in today's day and age people switch to Python, the python version bumps is such a wreck. Pip and Pipenv are 10 years behind the eco system. so many other better langs out there that are more uniform.
Because all ML and data tools are written by scientists in Python: PyTorch, TensorFlow
In C/C++/Fortran and imported by Python rather.