________________________________________________________________________________
I've always loved Scala and Scala 3 is shaping up to be extremely exciting. Love the new ADTs and braceless syntax: really gives the language a cleaner and more ML inspired feel.
No other conventional language (including OCaml, F#, etc) seem to have the effortless kind of power Scala does.
The combination of the extremely powerful type system with the "OOFP" paradigm is a great combination. OOFP is such a great paradigm and it's unfortunate that no other language has embraced it.
It's ironic that I love Scala so much because I don't particularly enjoy dealing with the JVM and all its complexities and problem. But Scala is so good that it mostly makes up for it.
Scala does get a lot of hate (especially around here), but I've never really understood why. There's only one language I like more (kdb+/q), but that's a hard sell, especially because of the cost.
Too bad there's not really a ML/AI ecosystem around (besides Spark, which seems to be losing favor partially because of its embarrassing performance), because using Python is such a drag. However, I do understand why non-CS/type theory people might find the language intimidating.
But to that I'd say, I've seen new junior (though very smart) developers with no Scala or FP experience get onboarded and write idiomatic Scala code within a couple weeks to a month. The short term cost is more than worth it considering the absurd productivity compared to Java/Kotlin/Python/blublang.
Compared to OCaml/F#/Haskell, the improvement is a little less stark, though still clear imo (especially when considering editor tooling and available packages).
Scala is weird. Sometimes it feels fluid and effortless but sometimes it simply cannot do things you would expect it to. For example, chaining custom methods. Why is it possible to do "arr.map(_.nonEmpty).filter(a => a % 2)" but not possible "arr.map(_.nonEmpty).myCustomFilter(_)"? In D thanks to UFCS and Properties you can chain all kinds of std and custom stuff together which looks more "efforless" and FP in my eyes.
Syntax aside I find Scala code hard to read due to how much implicitness it allows which stems from its "do FP or die" attitude which is again, weird for a language __designed__ to incorporate both OOP and FP. Should I mention that in this regard D does a better job at serving the both worlds?
And finally, which threw me off completely is a real world case where we had to implement a word-entropy based algorithm to process tons of textual data and it appeared to be not much faster than a Python version despite all the optimization we tried to do. Scala is fast but there are many faster languages including Java itself. Considering the fact that its compiling times force you to leave your seat for a cup of coffee, you'd rather think of investing into another hard FP friendly language like Rust which simply has a wider applicability area. But yeah, this is all speculative but if you are thinking about what next language to pick up, these things start to matter.
Where Scala fits nicely is a small team which needs to build some business logic on top of the existing Java backend stack quickly and have a reliable working system. This is where it really shines. Scala provides a nice layer for JVM devs who have a soft spot for FP, safe and elegant solutions.
There are no "special" methods in Scala. If you see something like "arr.map(_.nonEmpty).filter(a => a % 2)" in the standard library, you can use that syntax yourself too.
E.g. "arr.customlyFiltered()" is easy to do. Or even easier "arr.filter(myCustomFilter)". If something doesn't work out for you, please feel free to use e.g. scalafiddle.io and make it example, then you will be helped! :)
Scala is indeed not a high performant language. It is as fast as Java and compiles as fast as Java if you only use Java's features. However, then why use Scala in the beginning?
Scala is great for writing very maintainable and reusable code, especially with a big team and a lot of business logic. It is also great to "glue" things together like you can do it with Python. For these things it has more than sufficient performance in my opinion.
There is no straighforward elegant way of doing "obj.scalaMethod.myCustomMethod.scalaMethod.myCustomMethod" in Scala. I want to create a method and just inject it into the chain, no fiddling or ducktaping or using some black Scala magic.
"def myFun(a: String, b: Double): Boolean = {...}"
and then "obj.mapValues.myFun.forall(_)" or just anything similar.
And yes, Scala is exactly the language when you need safe and maintainable code with far less unit tests required than in Java. But you have to invest into it and sometimes it is simply not worth it. The Scala2 to Scala3 migration situation makes things even worse unfortunately.
There is a feature in Scala 2 that covers the specific thing you want: implicit classes. You can use them to add custom methods. It's clean and straightforward.
And it will be even easier in Scala 3, where this pattern is coded into an even simpler feature: extensions.
In the Scala 2.x this can be done with an implicit class.
It's a bit cumbersome and not very use-case oriented (mechanism over intention) and so Scala 3 has introduced extension methods[0].
This gives you exactly what you want.
Since Scala 3 is pretty much completely backwards compatible (and with the use of Tasty you can use Scala 3 code in Scala 2.x projects), so I don't see the migration as a big problem. The Scala team has spent a lot of work to make the transition as painless as possible.
[0]
https://dotty.epfl.ch/docs/reference/contextual/extension-me...
Thank you for URL. Just when I needed. Any idea when "Scala 3" can be production ready ?
From what I saw in the latest presentation by Martin, we can expect Scala 3 to be released in late Spring/early Summer 2021.
Part of the release testing is building the community release[0], which is a large part of the relevant Scala F/OSS code out there.
Whether you consider that to be production ready you have to decide for yourself, I myself am pretty happy about it.
Keep in mind that for now Scala 2 style code will still be compiled and you can use existing Scala 2 code in your Scala 3 project. The migration should be pretty smooth.
[0]
https://github.com/scala/community-build
It's not really black magic
https://docs.scala-lang.org/overviews/core/implicit-classes....
but I don't really see why having a filter as a method should even be encouraged.
Implicits are not encouraged in our code base and for a good reason.
This was just an example. It's not about filter at all but the general method chaining.
Here is example from D if I must:
real[] fun(int[] arr) {
return arr.map!(a => a.to!double / PI).array;
}
void main()
{
int[] arr = 100.iota.array; // [0, 1, 2, 3, ...]
real[] newArr = arr.map!(a=> a*2).array.fun); // [0, 0.63662, 1.27324, ...]
}
I don't know how you can chain a custom method "fun" to the output of the "map" in Scala without duck typing. Why is this not possible when all conditions type-wise are met? Why you can chain std methods like map, filter, reduce, fold etc. but not custom ones?
Ah, I understand what you mean now. You are looking indeed for the thrush operator. I think it should be built into Scala's standard library, but until that happens, you can use the mouse library or build it yourself. Here is an example:
// Need to define this once somewhere in your project
implicit class TrushExtension[A](anything: A) {
def |>[B](function: A => B) = function(anything)
}
// Your application code
def fun(arr: Iterable[Int]) = arr.map(_.toDouble / Math.PI)
val arr = 0 to 100
val newArr = arr.map(_*2) |> fun
newArr.foreach(println) // prints 0, 0.63662, 1.27324, ...
Execute or change the code here:
https://scalafiddle.io/sf/WAKhZtJ/0
This is maybe not exactly as convenient, but it comes pretty close.
"someA.someB.someC" becomes "someA |> someB |> someC".
If you would call it pipe operator, like it's called in many (Elm, Elixir, F#, OCaml) other languages that have it, people would understand you faster IMHO. It is even discussed for inclusion into JavaScript.
https://www.scala-lang.org/api/current/scala/util/ChainingOp...
Nice I didn't know that! I still like to add |> on top of it.
"foo |> bar |> baz" just visually parses easier than "foo pipe bar pipe baz"
Looks good! Thanks for the tip.
Scala is more strict than D for this use case. In Scala, if you want a function to be available to a certain type as if it was a method call, you need to be explicit about it. You have to declare an "implicit class" that takes the base type as an argument, and define the function as a method of the implicit class. You also need to ensure the implicit class is in scope. Once these conditions are met, you can use it as a method.
val t1 = MyType()
def fun(t: MyType, argument: Int) = argument
// can't do this yet
// t1.fun(42)
// In Scala 2 you use an implicit class to add methods to a type
implicit class MyEnrichedType(t: MyType) {
def fun(argument: Int) = fun(t, argument)
}
// In Scala 3 you use an extension
extension (t: MyType)
def fun(argument: Int) = fun(t, argument)
// now it can be done
t.fun(42)
Regarding your question about the stdlib methods being able to be chained: they are not special. They are defined for the type the methods return, so they can be used.
Rejecting all kinds of implicits and then complaining about Scala missing features is a bit unfair. "Implicit"is a single keyword, but not a single feature. Implicit arguments, implicit conversions and implicit classes are not the same thing. Fortunately, Scala 3 will clear this misunderstanding.
Assuming that your object is a list of tuples, then you can do:
val myFun = (a: String, b: Int) => b % 2 == 0
List("a" -> 1, "b" -> 2, "c" -> 3).map(myFun.tupled).forall(identity) // false
List("a" -> 2, "b" -> 4, "c" -> 6).map(myFun.tupled).forall(identity) // true
But I think that's not what you mean... are you maybe looking for the "thrush" / |> operator? Or do you have some example from another language that does a better job and show how it looks there?
> you'd rather think of investing into another hard FP friendly language like Rust which simply has a wider applicability area.
FYI Aaron Turon did their PhD on Concurrency+Scala, and ended up leading the Rust core team for more than enough years.
> Rust which simply has a wider applicability area
Rust is a great language but that's a pretty insane statement. The ecosystem is still tiny compared to the JVM.
Rust is a great language, and Scala is a great language, but they are different. They optimize for a different thing. Rust definitely rocks at high-performance, resource management, fine grained control over all the aspects of the program, at the expense of developer's time. Scala rocks at developer productivity and building abstractions, sacrificing a bit of performance.
However, I must say that Rust is also a very productive and quite powerful language (albeit IMHO not as powerful as Scala) and Scala is actually not that bad at performance either - probably better than a vast majority of other languages out there (albeit not as powerful as Rust or C++).
As far as ecosystems are considered - this is really hard to say. Rust has the whole C (and most of C++) ecosystem at hand plus a few really amazing Rust solutions like Cargo. Scala is limited mostly to JVM and JS, which are great ecosystems, but it is not true they offer everything. And there are still some caveats when using Java tools with Scala (e.g. profilers, debuggers or build systems).
Of course, general purpose languages have their limits and I'm not arguing that Scala can do everything. But it certainly can do a lot more than Rust (and its C/C++ interop) at the moment.
And I have no problem with the JVM interop. As long as you don't try to write Scala libraries that need to be consumed by Java applications, everything just works really.
A lot more than Rust? Do you count the whole embedded, low latency and scientific computing? As far as I like Scala, I don't think it is better in these areas than Rust. Even the async I/O libraries are more advanced in Rust than in Java.
> Scala is limited mostly to JVM...
No, it's not. You can call native (C) functions from the JVM.
Of course you can call C functions, but it is neither ergonomic nor performant. And good luck exchanging more complex data structures. Java can't use C structures directly, Rust can.
You still need a JVM, which what the GP is trying to point out (I think...)
That said; nowadays you have the Native and JS runtimes as possible targets as well. Scala.js works surprisingly good with very few caveats.
Out of context yes. I should have said that with respect to which language pick up next and which language has a bigger future potential. Rust is so omnivore and hits on so many critical aspects that it is hard to deny it. Given it had rich and mature ecosystem now how many would pick Scala? Those who prefer GC pauses and have a hard dependency on JVM? But even then you are pitted against Kotlin and Java itself. I would say quite tough times are awaiting Scala 3 and not only on individual level but on company level. It is still unclear how much the migration will take and whether it is worth it in the first place.
I don't exactly see how Rust can replace Scala where Scala is mostly used today. Or even Swift, despite Apple being more serious on the server-side roadmap lately and hiring a bunch of people from the JVM world.
Kotlin doesn't bring anything new and is full of ad-hoc design decisions, and Java itself is catching up (and even potentially surpassing it, see pattern matching for instance). I don't think it's a huge threat. Sure you'll see people who didn't really understand the point of Scala move to Kotlin because they just wanted a better Java really, and Kotlin does that well. But I think it's been clear that Scala shouldn't try to pursue this goal, for the past 5 years at least, after the hype around Spark cooled off.
I'm happy that Scala 3 is focusing on the biggest pain points. I'm also quite happy with the resources that have been added behind the tooling and ecosystem lately. The Scala center wouldn't exist if there wasn't a real demand for improvements in the mid to long-term future.
> compiling times force you to leave your seat for a cup of coffee
This is not the case in a codebase split into multiple modules that compile in parallel.
For me scala's feature set is absurdly big and there are always 10 ways todo things. Look at the spec of Scala. Its 200 pages long. That's insane.
I'd rather use Kotlin.
The Kotlin spec is...
275 pages:
https://kotlinlang.org/spec/pdf/kotlin-spec.pdf
And 796 pages for Java
https://docs.oracle.com/javase/specs/jls/se14/jls14.pdf
Yeah but the font ;)
> For me scala's feature set is absurdly big and there are always 10 ways todo things.
One of my favourite examples: of these 13 different ways of doing the same thing, only 12 are correct.
https://nurkiewicz.github.io/talks/2014/scalar/#/22
I'd say that holds about as much value as criticizing C-style languages for allowing all these variants
if(x) true
if(x)
true
if(x) (true)
if(x) {true}
if(x) {
true
}
if(x)
{
true
}
etc. In practice you use a linter to enforce a style and it's not a problem.
For the curious. The combinations follow out of fairly simple rules:
() and {} are interchangeable for expressions - altough {} can contain multiple statements and () only an expression.
x op y is equivalent to x.op(y)
Type abscriptions - x : Type - are optional and will be inferred if possible
{ case ... } is the pattern match construct and works similarly to a function with 1 parameter.
How is that a bad thing? It's the same in other languages and it is nice to have some flexibility.
E.g. lisp: use whichever type of parentheses you want
Or Kotlin: use short syntax lambdas fruits.filter(it == apple) or long syntax fruits.filter(fruit -> fruit == apple) or with annotated types: fruits.filter{fruit: Fruit -> fruit == apple}
Sometimes brevity is good for the reader, sometimes more details are good for the reader. Not everything is a nail.
The flexibility is perfectly fine when you're programming something on your own. But when you're in a team, or worse, in a large company, it starts to be a problem that everyone can use their own style. It's much easier to read someone else's code when you have a common code style.
And in Scala there are not only many different ways to use the standard libraries or to structure your code, there is a pure FP vs OOP-style FP schism as well. Because of this, you can qualify as a senior Scala developer for one job but only a junior in another.
I've been writing Scala for the better part of 8 years now and I have my list of gripes about the language, but this is one I have never understood.
Yes, there are different code styles you can use with Scala (standard vs infix notation, parens vs braces, etc) but all of that can be standardized with code formatting tools.
In terms of FP vs OO style I don't think that is different for any other language. No matter which programming language you choose you have to make decisions about what sort of patterns you want to use in which scenario and enforce that across the team. I have seen many Java projects where a relatively small codebase has approximately every GoF pattern implemented somewhere (and a few novel patterns just for good measure).
The opposite. If I write code on my own just for myself I use the highest level of brevity.
But if I write code for a team, then in some places I will use explicit type annotations and variable names to aid people unfamiliar with the code to understand what's going on.
What you are saying is pretty much "it's easier when everyone only uses nails, because then all I have to bring is a hammer". I think it's good to use screws sometimes.
However, your second point I agree with. OOP vs. FP is a different story. This is about paradigms not about mere syntax. So here, a team must be aligned, which can be a challange when using Scala. It's not a language for corporate drones.
If it was so simple as just using explicit types :)
And this sort of flexibility is exactly why Scala is hard to read. Perl is another example of a great language that allows you a lot of flexibility. In fact, the ability to use just any symbol for a method is the worst thing about Scala. You quickly realize it once you start using Scala libraries some of which basically introduce you to a new Scala based DSL. This quickly becomes a nightmare to work with if several such libraries are used in the project.
Have you used Scala before? I used both perl and Scala and I had to laugh when I read what you said.
Yeah, Scala makes it possible to write cryptic DSLs. I just don't use libraries that do that, but there are not many such libraries anymore anyways, that was mostly abused in the early days of Scala. Now, 15 years later, that's almost non-existing anymore.
But not having symbolic characters in method names is just horrible. Here is Java code that calculates some datetime:
LocalDateTime started = LocalDateTime.parse("2018-03-22T19:00:00")
LocalDateTime finished = start.plus(Duration.ofMinutes(30)).plus(Duration.ofMinutes(15).multipliedBy(4)).plus(Duration.ofSeconds(45)).plus(Duration.ofMinutes(4).multipliedBy(3)).plus(Duration.ofSeconds(30))
Here is the exact same calculation, but with Scala's "symbolic" characters:
val started = LocalDateTime.parse("2018-03-22T19:00:00")
val finished = start + 30.minutes + 4 * 15.minutes + 45.seconds + 3 * 4.minutes + 30.seconds
I don't need very long to know what I find far more readable ;)
"I just don't use libraries that do that" -- sounds like you found a solution :)
I agree that "+" looks better than ".plus" but this is a matter of taste (hello Ada).
Yes, I use Scala at work but I can't call myself a seasoned Scala dev, true. On the other hand, just every expert in this or that language will always have a counterargument for you and a solution to the problem. Because he IS an expert! The thing is, not all people are experts or even will be. Most people want a tool that doesn't get in your way, easy to pick up and deliver the results. That is why Python is so popular despite being slow, inefficient and basically a glue for C/C++ libraries. Hell, even for interfacing with C Python loses to Lua. But all these shortcomings didn't matter in the end. Python2 → Python3 didn't kill the language either. I am afraid Scala's error margin is not as big.
Scala has certainly weaknesses. But being able to use symbols in method names has both pros and cons. You saying that it's the worst thing of Scala is just very exaggerated, at least from my subjective perspective.
And yeah, you are totally right. Most people indeed choose the way of the least resistance and don't want to spend much time learning a language - for good reasons. Scala will never be as popular as Python or Javascript. It is a language for people that are ready to invest more time and dedication to get a higher productivity after some time. Not everyone wants to do that and many people find it too hard. I think the Scala community knows that. :)
> But when you're in a team, or worse, in a large company, it starts to be a problem that everyone can use their own style
I was in many failed or semi-failed projects and code _style_ was never _the_ problem. Yes, people do use slightly different styles, but style is really easy to enforce - there are tools to do it automatically. And even if somebody misplaces a brace or writes `map(_.length)` instead of `map(x => x.length)` this doesn't impact the readability as long people in your team know the language. If a project fails to meet the deadline because of the code style it is not because of people using different styles but because of developers bikeshedding about the code style in code reviews instead of doing real work.
The Scala ecosystem has tooling to enforce code style across teams.
Your second argument is true, Scala is a big language, and there are three main paradigms: better Java, OOP+FP, and pure FP. It's similar to how C++ is used, for some people is just like C with some improvements, but there are people using lots of C++ features.
That is when company wide style guide helps.
> How is that a bad thing?
This doesn't quite answer your question, but, the Python community takes seriously the idea of _There's Only One Way To Do It_, as part of their philosophy on complexity.
https://wiki.python.org/moin/TOOWTDI
https://www.python.org/dev/peps/pep-0020/
Seriously?
How to do data structures:
1. just put everything ad-hoc into a dict or a list (the "PHP" way :D)
2. use a namedtuple
3. define a class
4. define a dataclass (preferred) but works only if your Python version is recent enough
In Scala? Just use case classes and this is the only recommended way (#1 is very impractical so noone does that, #2 doesn't exist, #3 possible, but impractical when you have #4 in all versions).
How to map a collection in Python?
- Start from an empty one and add mapped items in a loop
- Use map + lambda
- Use list comprehension
- Update all items in place with a loop
Is it really any better than in Scala?
Scala goes quite far in the flexibility, maybe too far. Agreed.
But is python really so good? I'm not a heavy python user, but there are loops and list comprehensions and map. There are also optional type annotations now, should you always use them, or not? How about "a is b" or "a != b"? How about environments and build tools...
I think that go might have been a better model student.
This is like saying, airplanes are too complex. There are so many knobs and buttons. I'd rather just walk or ride a bike everywhere.
Scala is a very powerful, very expressive language. There are some features which you can just choose to leave out. If you do, you end up with a very clean, concise, and powerful language that makes you really productive.
I've literally had moments where I made my algorithm 5x faster just by adding 4 letters: '.par' in front of an operation. Instantly it got parallelized without my having to do anything, and the processing time got cut 5x.
Its more like saying: I need to cut some paper ( i need to program): Swiss army knifes (scala) are too complex, I just need a pair of pair of scissors (lisp / other languages).
Scala really isn’t that complex though. It’s actually pretty intuitive.
You CAN write bad and hairy scala code just like you can in any language. But that doesn’t mean the language is to blame.
And that's why there are far fewer pilots in this world than stewards.
Having switched from Scala to Python recently, I must say Python has probably 5x more ways to do things than Scala and I can't see people complaining on Python's complexity.
This is something that really surprised me, because "one obvious way " was announced to be a part of Zen of Python.
and 0 way to do a lot of things
IIRC one of Odersky's stated goal for Scala 3 was to address exactly this issue. That being said, it is developed in a research setting so I guess even if they restrict these features now, there will be new ones added over time...
Scala has always been an opinionated language. The main difference is that this time is holding opinions against itself. Previously, opinions were held against Java, that's why there are features such as case classes, traits, and immutability by default. They answer to Java problems.
Scala is now mature enough that some issues have emerged. Typeclasses are cool, but programming typeclasses in Scala is a bit clunky. The type system is very powerful, but it could be even more powerful and that would actually help people write simpler code. And there are a bunch of unused features that could be removed. This is what Scala 3 is about.
Scala 3 tries to be more opinionated and goes quite some miles (kilometers?) to make the language more regular and simple.
The implicits are gone, for example. Kinda crazy, since it was Scala's banner feature for so long. For the primary use-cases of implicits (extensions, type classes, conversion, ...) there are now language level concepts to make these use-cases accessible.
"Intent over mechanism" I think Martin calls it.
Yep,
Just give it a couple of more years.
I am skeptical about the novelty of Scala's "OOFP" as you put it. It looks essentially the same as the original anonymous classes or SAM types in Java. Function values are objects with a single method and methods themselves are not first-class, but are wrapped in objects as required. The most important part of OO are the first-class modules; and in the literature, arguably more elegant examples of unifying first-class modules and FP exist, for example by starting with FP and adding objects as records containing functions, this way both "objects" and methods would be first-class. Subtyping also adds significant complexity to the language and weakens type inference.
Scala is, first and foremost an OO language which embraces subtyping, class hierarchies and imperative programming, see for example its own collections library. As Java gets more and more FP features, there is a danger it will struggle to differentiate itself.
Your comment makes sense if we think of FP as being about "function values". In a world where all sorts of programming languages have function values, this is not a strong enough characterization anymore (if it ever truly was).
Functional programming is about programming with _immutability first_. So OOFP is about immutable objects. I go even further to say it is about objects with _immutable interfaces_, but the implementations can be imperative as long as _encapsulation_ hides the mutable aspects from the user of the object. This is what you see in Scala's collections library: we use mutability inside the implementation but we expose an immutable API.
I gave an entire talk on that very topic at Scala Italy in 2019:
> Functional programming is about programming with immutability first.
I completely agree with this. So perhaps I am being rather hard on Scala.
_It looks essentially the same as the original anonymous classes or SAM types in Java._
Decompiling a Scala .class file shows just how true this is.
I really enjoyed writing Scala code. I'd agree with your evaluation with respect to the expressiveness and power provided by the language.
...but the tooling, at least at that time, was terrible. SBT was terribly overcomplicated, and full of foot-guns. I spent far too much time debugging dependency collisions, issues created by so-called "autoplugins" and other nuances that had nothing to do with getting real work done.
Now I'm writing a lot of Go. I'll admit, it's not nearly as fun to write. There's a lot less creativity and expressiveness -- though I think this is probably a good thing. But it also feels like I'm getting a lot more stuff done, and the tooling is amazing.
I have the same feelings about Scala. It was fun to write Scala code and it was inspiring, but the tooling was terrible and there were way too many possibilities to express the same thing. Scalaz was essentially a language in a language that had nothing todo with "normal" Scala.
Now I'm using Kotlin. While a miss a more advanced Scala feature from time to time, I feel more productive because I don't have to think so much about the language itself. And Kotlin is much easier to introduce, plus the tooling is way better.
SBT is a fractal of awfulness and I have no idea why anyone ever uses it. But it's never been required. Just use maven and get on with your life.
Yes, SBT is just terrible! It is the reference for usability nightmare in the category build tools.
cbt and mill are just obvious scala code
How's their IDE integration? (Though TBH I'm coming from a starting point of "Maven isn't broken, there's no need to fix it")
for mill intelij and vscode via the metals extension
I use maven to compile Scala.
> Too bad there's not really a ML/AI ecosystem around (besides Spark, which seems to be losing favor partially because of its embarrassing performance)
Compared to what? If you mean engines like Presto, I would say they have different use cases, plus Spark SQL puts them roughly in the same ballpark. Genuinely curious though, as a Spark user that is always interested in alternatives.
kdb+/q performance smokes Spark. Actually. most native code on a single machine smokes an entire Spark cluster.
> Too bad there's not really a ML/AI ecosystem around (besides Spark, which seems to be losing favor partially because of its embarrassing performance), because using Python is such a drag.
I think it's not that bad. Sure, it loses to Python, but that's some tough competition.
Personally, I find that smile[1] covers most use cases.
Breeze[2] also has a lot of love, but I'm not much of a fan of the `UFunc` abstraction.
I'm also really excited about polynote[3]. It's still a bit green and only supports Scala 2.11/2.12, but it's such a joy to be able to use Scala in notebooks :). You also get Python interop, in case you need to use some specific python library.
P.S.: I hope that Scala's ML/AI libraries other than Spark keep growing, because "embarassing" is a nice way to describe it's performance on everything that's not a "really-huge dataset".
1:
https://github.com/haifengl/smile
2:
https://github.com/scalanlp/breeze
3:
https://github.com/polynote/polynote
Many if not most Scala data/ML libraries are one-person projects which are either abandoned or have compatibility issues. It's a barren land and loses to any similar Python library in terms of functionality. It's good that Java ecosystem can back Scala up otherwise it would be pretty sad.
When Scala, IntelliJ and SBT cooperate the language is so pleasant to work with.
There are some puzzling omissions though.
For example slice method is missing step.
Coming from heavy data munging Python this really bites.
Sure would be nice to have some syntactic sugar for slice.
Some seemingly simple tasks have no one way of solving them.
Let's take parsing JSON. Trivial in Python, painful in Scala. (almost the reverse in Python which has painful XML parsing and Scala's built in support for XML)
So far the easiest JSON library has been Li Haoyi's uPickle:
https://www.lihaoyi.com/post/HowtoworkwithJSONinScala.html
Still, it does not parse JSON where objects have uknown arbitrary value types. Arbitrary value types are extremely common in real life JSON.
Due to pattern matching you see people suggesting you write your own parser! Sure it can be done, but then the next thing you'll be rolling your own crypto...
> So far the easiest JSON library has been Li Haoyi's uPickle:
https://www.lihaoyi.com/post/HowtoworkwithJSONinScala.html
> Still, it does not parse JSON where objects have uknown arbitrary value types. Arbitrary value types are extremely common in real life JSON.
Looks like ujson, which the article you point to talks about, is what you're looking for. uPickle is a layer on top of ujson for statically typed stuff, but ujson is working with raw JSON values, of arbitrary types.
Thank you! I will have to look into ujson deeper.
You can in fact mix typed and untyped parsing, by having a `ujson.Value` field in the middle of your typed case class or collection. It just works
Thank you Li!
`ujson.Value` is in chapter 8.1.1. of your excellent Hands-On Scala book I've bought.
Turns out for arbitrary JSON you can do this:
val rawData = read[ujson.Value](JSONstring)
For JSON where you know most of the structure but have mixed types for object values:
https://jsonplaceholder.typicode.com/todos
val todoData = read[Seq[Map[String,ujson.Value]]](todoJSON)
Why wouldn't you just use Jackson for JSON? It makes parsing JSON trivial on the JVM.
> only one language I like more (kdb+/q), but that's a hard sell, especially because of the cost
J? It's not quite the same, but is OSS.
>Too bad there's not really a ML/AI ecosystem around (besides Spark, which seems to be losing favor partially because of its embarrassing performance), because using Python is such a drag.
Even if Spark might not be the best tool for ML, it is still the tool to beat for data processing and custom ETL.
> considering the absurd productivity compared to Java/Kotlin/Python/blublang
I have not see more unproductive language than Scala.
Scala is awesome. It gets a lot of hate around here because many of the users of this site are Super Intelligent and Could Have Done It Themselves, Only Better.
Scala deserves all the hate. It might be a fun language for a throwaway pet project but I'd never recommend it in an enterprise setting.
It's an extremely unproductive language because it's too flexible. There are so many different ways to do the same thing which introduces unnecessary complexity.
Yet plenty of big companies manage to ship software written in Scala just fine.
Very, very excited about Scala 3. Here are some of the things I'm looking forward to using:
Scala has already been a wonderful language to work with professionally with many great tooling (linting, automatic migration, 2 good OSS IDEs), and I think Scala 3 will certainly push the state-of-the-art forward in many departments in the coming years.
[1]:
https://www.cs.purdue.edu/homes/rompf/papers/rompf-icfp15.pd...
[2]:
https://twitter.com/not_xuwei_k/status/1323643312230772737
In my company (10k+ devs) Scala seems to be losing steam. All the enthusiasts have moved to Go/Rust, while the pragmatics are staying in (moving back) in Java. No new major project is being built in Scala.
Does anyone share this experience?
Medium company (low hundreds of devs). My team is starting some new projects in Scala, which will be the first Scala at the company.
So far it’s going okay. But I personally am surprised at how disinterested most folks are in learning anything new.
> am surprised at how disinterested most folks are in learning anything new.
same here, we don't do Scala but it's basically impossible to get people here to learn something new. Introducing a new language (like Scala,F# etc) would be impossible.
Why did your team choose Scala , assuming you are bound to the jvm what sold Scala over kotlin or Java.
The tl;dr is we had absolutely no good business reason to choose Scala, and very few technical reasons. It is mostly a developer happiness gamble.
A secret ulterior motive of mine was that Scala has bad support for Spring. Every other Java service at the company uses Spring, and I abhor it, so if I can drive adoption of Scala on my team then we can avoid Spring and also provide pressure to make all our shared JVM libs Spring-agnostic (which I believe is a good move in general).
Scala was the first serious attempt to break Java's stranglehold on the JVM that made any impact. A lot of people got very excited by it.
But then Kotlin, Ceylon, Clojure and others started to steal it's thunder and now it's just "one of the pack".
In fact, I'd say Java itself has stolen back from all those other languages to a large extent, with its language enhancements since v8.
With the exception of Ceylon (because it's basically dead), there's actually a really nice gradient on the JVM these days:
Clojure is a nice, expressive, dynamic language that is really great for messing with arbitrarily-shaped data.
Java is a workhorse language that has been powering businesses small and large for decades.
Kotlin is a modern spin on Java, and the degree to which they share functionality (with differing or identical semantics) over time will be interesting.
And then Scala exists for all your meta-type-wankery needs.
It seems natural that Kotlin/Java would be most popular over time, with Clojure and Scala hanging out with small amounts of market share.
My company uses Java for most things.. some python for scripting. There are a few of us who use Scala because of the work we do (Spark). I have to say that I prefer it over Java, and Spark is great for what we use it for. The functional features allow you to be concise and require a bit of creativity which makes it fun to work in. However, I doubt that I would use Scala outside of Spark. I think it's mostly because other typical frameworks that get used are Java-based so I would just code in that. That seems to be true of most of what I've done professionally: the framework or platform that makes your job easier (or even necessary) usually seems to require some specific language. (e.g.) When I coded a lot in IOS, it was Objective-C and then Swift (liked them both); a lot of web frameworks use Java or Python; front-end dev : javascript. At some point, they all just feel the same to me, to be honest. Some I like more, some I like less.. but the process is usually about the same. Clearly there seems to be the theme in the industry to move general-purpose development closer to machine-code (away from JVMs and the like).. Go, Rust, Swift, Lua, Dart are some examples. I think this is generally a good thing. At some point, Spark alternative will appear that will probably be written in Rust, if I had to guess... (Vega? or whatever it's called)
My experience as a freelancer is that the two camps are:
- enthusiasts who will stay with scala
- others who think they should have used kotlin/java from the beginning.
The latter camp is bigger and growing.
In my experience people just moved to Kotlin. Hardly seeing any new projects in Scala, except maybe for Spark and such. It is mostly just maintenance at this point.
Enthusiasts moving to Rust (and Haskell), sure. But Go? That's definitely a language for the pragmatics, right alongside Java.
I think the big gap between releases was part of the problem. 2.13 took forever and frankly there haven't been any must-have new features since 2.11. So Scala is no longer new and exciting. But it's still the best at what it does.
I mean, if Rust had higher-kinded types, and the established library ecosystem, and the IDE support that Scala does, then I'd move to Rust too. But it doesn't. When you have a large codebase and need to manage cross-cutting concerns, the only thing that can do that as well as Scala does except Haskell (or more obscure options like Idris), and the state of Haskell tooling is miles behind Scala. I'm not attached to Scala for Scala's sake, but it's still my first choice.
What I'm seeing is that Scala devs I know have started using Kotlin (including me). It is a much more pragmatic language that doesn't try to reinvent the wheel, like Scala does.
In my environment Kotlin has won the better-Java use-case quite thoroughly.
Same experience here. JDK11 and beyond added some more enthusiasm to base Java. If Loom lands I think a lot of people will also lose interest in Kotlin as well (because AIUI a big draw for Kotlin is the async story).
I completely disagree. The best things in Kotlin has nothing to do with coroutines, it is the extension functions and the soon-to-be released typeclasses (multiple receivers). Java is lagging behind, while Kotlin gets pragmatic and useful features very frequently.
I have some friends/colleagues who use kotlin in personal projects and they enjoy it. But I've never heard them talk about those features. And they certainly aren't enjoying it now because of soon-to-be-released features.
But I am interested to learn more. Do you know any good resources?
I think the soon-to-be-released features anxiety is applicable for any language, it is not unique to Kotlin. A good resource for this is the Kotlin Koans for example or the Kotlin STDlib itself. I also maintain some OSS projects that use these things extensively (pun intended). What's also present in the language is the multiplatform functionality. You can't do this in Java and you probably never will be able to do it. With this I think Kotlin is one of the two languages that's capable of isomorphic behavior all across the board (you can write everything in Kotlin).
Most Kotlin devs I know are not particularly interested in coroutines. There is a lot more to like about Kotlin: native function types (instead of functional interfaces), unified type system, bottom type, Unit type (instead of the special case "void"), extension methods, no checked exceptions, sealed classes ...
Scala 3 is a huge change. As someone who used Scala for years, but always wondered whether they can retain their users with competition from Kotlin, I'm extremely curious what this will mean for Scala in the long run.
I'm skeptical, because the cost to migrate is big. But I'm hopeful, because Scala is a language I like to write code in.
Edit: why am I skeptical? I know scala 3 is largely meant to be backwards compatible. But scala has always been a language of many styles ("java without semicolon" vs "haskell compiled with scalac"). All this new syntax in scala three adds a whole new dimension to this issue.
On the JVM, I think Scala won't have any major issues it is just another guest language and isn't used to sell InteliJ licenses.
Kotlin is chaining itself to Android, it will rule there thanks to Google's sponsorship, on everything else it is just yet another language to chose from with a weaker eco-system, used to sell InteliJ licenses.
> used to sell InteliJ licenses
Though IntelliJ has amazing features for Java/Kotlin: it comes with a free version that packs most of 'm. I think saying it's all to sell licenses is a disingenious, lots of longstanding Java pain points get addressed by Kotlin in a really nice way. It has a really strong webdev ecosys building up, and comes with a rather interesting feature set as language itself. I'd say its good "typed Ruby" (OO at the core, FP where it makes sense, very expressive, dont type too much).
Scala's issues on the other hand stem, I think, from it being multi-paradigm. Where Kotlin is OO-core with FP where it makes sense, Scala is both OO and FP at the same time which makes it messy.
Frege explored being full FP on the JVM, but looking at the repo[0] it did not get much traction.
[0]:
https://github.com/Frege/frege
Not at all, JetBrains are the first ones to admit it, also why they decided to stop contributing to Eclipse and merging the Kotlin plugin into the InteliJ source tree.
> Kotlin support for VSCode or other IDEs is not on the roadmap for the Kotlin team. Community initiatives in this respect are welcome.
--
https://kotlinlang.org/roadmap.html
> The next thing is also fairly straightforward: we expect Kotlin to drive the sales of IntelliJ IDEA. We’re working on a new language, but we do not plan to replace the entire ecosystem of libraries that have been built for the JVM. So you’re likely to keep using Spring and Hibernate, or other similar frameworks, in your projects built with Kotlin. And while the development tools for Kotlin itself are going to be free and open-source, the support for the enterprise development frameworks and tools will remain part of IntelliJ IDEA Ultimate, the commercial version of the IDE. And of course the framework support will be fully integrated with Kotlin.
--
https://blog.jetbrains.com/kotlin/2011/08/why-jetbrains-need...
Thanks for your reply. You have a point, but there is to me still a difference between a "scratch-itch language created and a good biz model on top", and a "biz model that required a new language".
There is some legal stuff going on as well: Java was being monetized by Oracle and Google needed a way out. This helps Kotlin a lot imho. When then free-Java case totally lost, Google allows all to move to Kotlin and IntelliJ has the code translator tool (and will prolly get bought by Google at some point).
Google screwed Sun and had their opportunity to buy it.
The Java ecosystem has dozens of JVM implementations. None of them have ever had a problem with either Sun or Oracle.
Only Microsoft with their J++, and Google with their actions fragmenting the ecosystem for Java developers.
Microsoft learned their lesson and are now a OpenJDK contributor.
Time will come for Google as well.
Switching to Kotlin doesn't remove the dependency on the Java world, unless they plan to rewrite everything in Kotlin/Native.
Kotlin has significant support from the largest Java ecosystem: Spring. In my experience it's also much more compatible with Java/JVM libraries. In Scala that's perfectly possible but leads to very un-idiomatic code, in Kotlin you barely notice.
> In my experience it's also much more compatible with Java/JVM libraries. In Scala that's perfectly possible but leads to very un-idiomatic code, in Kotlin you barely notice.
I think that's actually reversing as Kotlin and Java diverge. E.g. interop with Java Optionals is very easy in Scala (and they obviously correspond directly to Scala Options) whereas it's harder to make them fit with Kotlin nullable types. And Kotlin's "suspend functions" are unlike anything else and mean you have to understand a concept of "inline" that changes the semantics of the function it applies to; e.g. calling a Java function that takes a callback from an async function is straightforward in Scala, but not so much in Kotlin.
Scala has an issue with the concept of "idiomatic code". I'd say when it comes to "idiomatic code", Go and Scala are at the opposite sides of the spectrum.
Spring supports everything that helps their bottom line.
Where are the Groovy, Scala, Clojure support nowadays?
Groovy is still alive within the context of Grails (which is basically a very nice wrapper around Spring) and Micronaut (which is a direct competitor to Spring). Micronaut officially supports Java and Kotlin as well.
I was referring to the way Spring used to sell that they supported those languages, just like they do with Kotlin now.
Scala 3 supports braceless syntax, aka significant indentation, aka "YAML is to JSON as Scala 3 is to Scala 2"
https://dotty.epfl.ch/docs/reference/other-new-features/inde...
There was quite some debate on this, going on strong.
I consider this a totally useless "innovation"! As if Scala didn't have already enough possibilities to express things, no we are able to leave out braces, what makes scanning code even harder and will lead to an ugly mixture.
I would accept it, if they would decide to drop the traditional syntax. But adding yet another style, just for the sake of it, is waste of resources (of the Scala creators and Scala users).
There is a good article about the "strangeness budget" of languages, and I think that Scala doesn't spend it wisely.
https://steveklabnik.com/writing/the-language-strangeness-bu...
Yuck... personally I'd want a Lint that would disallow many such syntax differences from Java:
- force use of ()
- force use of ;
- etc...
Yeah there are certainly some Python 2/3 fears for Scala 3, but the plan and upgrade path seem much more thought out
The crucial point here is if Scala 3 maintains backwards compatibility with Scala 2.x. If not, it's effectively a new language and then the game becomes which of the existing scala code bases will switch over and when. There's a large amount of existing Scala code out there and embarking on a migration of that stuff is going to take very long. We're talking many years here potentially. The few Scala projects I've dealt with, even updating to minor releases of Scala was a big deal.
The python 2 to 3 transition took well over a decade. You have similar discussions in what remains of the Perl community around v5 and v6. IMHO what Oracle has been doing with Java in recent years is impressive in the sense that they provide a credible upgrade path and put a lot of effort into that while still adding non trivial new language features. But it has a price of added complexity or compromises with new features and unfortunate legacy features that are not going to way. Javascript and typescript have the same issue. Javascript has a lot of weird legacy features and typescript preserves backward compatibility with that weirdness and attempts to engineers around that.
I'm currently using Kotlin across Android and Server (mostly) with an intention to also use it on IOS and Web very soon. Its multi-platform capability is increasingly a good selling point and I love how that ecosystem is booming in the last year/months. I'm definitely an early adopter of multi-platform but this seems to be one of those things where it seems like a bet worth making at this point.
Kotlin is of course a much younger language so it does not have a lot of legacy yet burdening it. Yet, Jetbrains seems to be good at managing new features while minimizing disruption, dealing with necessary deprecations, and generally not breaking compatibility. Their experience as tool makers gives them a unique edge over a lot of other languages.
Arguably Kotlin emerged as a rejection of Scala to replace Java by Jetbrains: they considered it and dismissed it as an option and then embarked on a journey to create their own language. I think the success of the language (relative to Scala) seems to indicate that that wasn't a bad choice. Scala intended to do many of the same things but it just never caught on with people actually doing those things to the extent that Kotlin seems to be catching on. The transition from Java to Kotlin is a lot less of a hard sell than the transition to Scala was. Though I know plenty of people that stubbornly stick with Java still. Of course early adoption in the Android community was a big deal. But then you could argue that that same opportunity was there for Scala to take and I don't think much serious effort ever was put in that notion. The need for something else than Java was quite big there and a big driver for this. All Kotlin did was provide a credible upgrade path to Java developers and the Android community jumped on it because it was easy to switch and obviously better. You see the same happening in the Spring community which is increasingly Kotlin centric.
Meanwhile Scala seems to be bleeding mind-share to more pure (for lack of a better word) languages Crystal, Elixir, etc. or more system oriented languages like Rust, or indeed Kotlin. It's a bit of a repeat of what happened to the Ruby community a few years ago. Certain types of people were very passionate about that language for a while and then moved on.
> The crucial point here is if Scala 3 maintains backwards compatibility with Scala 2.x.
The Scala team is keenly aware. Scala 3 can use Scala 2.13 libraries - in fact the 3.0 stdlib is the exact same .jar as the 2.13 stdlib - and a future version of 2.13 will be able to use scala 3 artifacts as long as it doesn't use 3+ only language features.
More details at
https://scalacenter.github.io/scala-3-migration-guide/docs/c...
Except Macros. So a lot of stuff.
Yes, but they're mostly constrained in libraries and there's a way to publish them so both 2.13 and 3 can consume them. You can expect "core" ecosystem libraries to be cross-published and most others don't depend on macros.
Note that most things you'd need macros for in 2.13 have language support in 3, so for user code - if that even uses macros at all - the upgrade path is fairly straightforward.
Is there a document with the differences between 2.13 and 3?
https://dotty.epfl.ch/docs/reference/overview.html
and associated pages.
I was also asking me for such details. Thanks.
I have a pretty large codebase written in scala 2.12 or so (whatever the version was around late 2018). I'd like to migrate it to scala 3. Any ideas how hard it'll be?
Start with migrating to scala 2.13, scala 3 can run all non-macro 2.13 code that doesn't emit deprecation warnings and can consume most 2.13 libraries. The widely-used ones it can't will get updated.
For more information see
https://scalacenter.github.io/scala-3-migration-guide/docs/c...
Pretty easy unless you were writing custom collections or custom macros, IME.
Just want to give Martin Odersky a shout out - what a legend this guy.
62 years old, going strong.
I'm not even in the java ecosystem but Scala strikes me as the most likely language to do something fresh and interesting on top of an existing ecosystem in the next few years, which is terrific news.
There are many reasons why I respect him, but the fact that he laid out a time plan for Scala 3 already 2 years ago and then _kept_ the timeline is mind-boggling.
The Scala language is great but the ecosystem sacrifices simplicity for pure FP. Scala has always suffered from complexity problems due to libraries. For example, using Http/JSON/Database libraries have always been a lot harder than they need to be due to a die hard approach to pure FP.
Scala 3 looks pretty good, but I just can NOT see myself using Scala again, after having worked with it for 4-5 years at work and on personal projects. I write mostly/reasonably FP code without using Cats/Category theory. I’ve recently moved all my code from Scala to Kotlin and I’m loving it, found the perfect balance with Kotlin.
> For example, using Http/JSON/Database libraries have always been a lot harder than they need to be due to a die hard approach to pure FP.
Hu? There are plenty of alternatives to the pure FP ecosystem.
HTTP: Dispatch, Requests-Scala, literally any Java client if that's still too FP for you
JSON: uPickle, Play JSON, or simply Jackson
DB: ScalikeJDBC. Quill and Slick do a lot more but are not particularly die-hard FP either.
Same here. Kotlin is a pragmatic language and it is a much better fit for the "Turbo Java" role that Scala originally wanted to do.
I have downloaded this big open source project which uses spring and ibatis and have everything which OOP offers.
For learning Scala, I am trying to re-write it using play framework. Case classes do offer reduction in boiler plate code. But write now feeling overwhelmed about how to map all this deep inheritance tree, factories etc. from OOP world to functional way of doing things.
I hope someone write side by side mapping of doing things in Scala 2 vs Scala 3.
Nice to see Scala 3 finally making the first steps to release.
The most important thing that no one seems to mention is that Scala 3 is not compatible with Scala 2. This means that they are creating competition for themselves. It is the Python 2 vs 3 problem all over again.
I never quite got what made python 2 -> 3 so much more complicated than say php 5 to 7.
I heard nobody complain the time I was still active in the PHP world (lucky me right?). Yet the Python2 code just kept lingering everywhere.
Scala can be nice but up until now compilation is way too slow to live with. Hopefully this will make it more palatable, but I've moved to Kotlin already.
I've been working with Scala for 7 years now. Compilation speeds have been getting better throughout the 2.12.x/2.13.x cycles. Compilation speeds for Scala 3 are looking very good too.
Nice. Looking forward to start using all the 'intention over mechanism' features that Dotty introduced.
Scala 3 making progress is great, just like progress on the Scala 2.13 camp:
Scala on Track to Support Java 9 Module System
https://github.com/scala/scala-dev/issues/529
Scala 2.13 on track to support Spark