💾 Archived View for dioskouroi.xyz › thread › 29429989 captured on 2021-12-04 at 18:04:22. Gemini links have been rewritten to link to archived content
-=-=-=-=-=-=-
________________________________________________________________________________
I have also seen some videos on Youtube sponsored by Redis, they want to hammer the point that they're not just a caching tool, but a more serious DB.
https://www.youtube.com/watch?v=OqCK95AS-YE
Can someone speak more to this? I've also seen it stated that it's possible to use redis for durable data storage, is that really the case? What do you lose out on?
From my experience it works pretty well, the issue is that the idea that it's temporary storage is so prevalent that I wouldn't risk it.
From what I saw, people think two times before doing `truncate table` but not that much with `FLUSHALL` in redis...
Also, I don't really see where the usecase would be as a permanent datastore compared to something like postgres.
They have two different approaches to persistence, with different tradeoffs:
https://redis.io/topics/persistence
Either would have the generic disadvantages of:
- Dataset can't exceed available memory
- The general disadvantages of NoSQL vs SQL around queries, transactions, etc, if you're expecting relational type functionality
Redis supports clustering and sharding:
https://redis.io/topics/cluster-tutorial
There are also managed solutions that implement Redis with partial flash storage. Redis Enterprise and AWS ElastiCache both do this.
> Dataset can't exceed available memory
From my understanding, they do support sharding
Sure, but still limited to the total amount of memory of all machines. You can't page in/out from storage.
You can store Terabytes of data in Redis. Just google it.
Yes, in Terabytes of RAM, or some separate commercial offering with flash. Not sure I get your point.
I have found this to be useful:
https://github.com/fpereiro/backendlore
It is more about writing backends generally, but in it he uses Redis as a primary database and walks through durability and scalability.
Broken and laggy scroll, hijacked back button. Why can't we have normal web pages again?
Opened in FF and Edge, and am not experiencing a hijacked back button, nor laggy scrolling.
Hmm, I am not experiencing this though I am using FF with some anti-click jacking plugins...
I'm just using uBlock Origin, what do you use?
I use the following, in addition to NextDNS.
- uBlock Origin
- Privacy Possum
- DuckDuckGo Privacy Essentials
- WebGL Fingerprint Defender
- Font Fingerprint Defender
- AudioContext Fingerprint Defender
- CSS Exfil Protection
If I were to build a Hacker News clone, I'd probably do it in a seriously old-fashioned way:
So basically, what the current HN is sans lisp.
Yeah, pretty much.
An actual clone, no real value add. But it'd be fun to do, regardless.
There’s a full feature clone in Python on GitHub, in case anyone actually want features, not just tech learning:
https://github.com/datatau-net/datatau-net
You know... Hacker News is built with a pretty-obscure framework IIRC that is little-used elsewhere, and we're all about open-source. It would almost make sense to reimplement HN in something more common and then open-source it so that regular developers could make improvements.
I'm solidly in the 'if it ain't broke don't fix it' camp on that one.
I'm solidly in the "Hacker News is the Craigslist of web forums" camp. It may not be broke, but there is a lot of room for improvement...
I have to be honest, the 'craigslist of web forums' is the entire appeal for me. It's simple, fast, low clutter and in my opinion benefits from great moderation and a general involvement from the community to avoid treating it like a gutter. I could be suffering from limited imagination, but I'm not really seeing major opportunities for improvement. Change? Yes, for sure, lots of ways to change the site but I don't know how they would result in an outcome of improved community or experience.
I actually love the design of Craigslist. it works well enough for you to find what you're looking for (if it exists), but it does not invite you to linger afterwards.
you can’t fix the redpilling morons who were exiled from Reddit, though.
Half the posts are political, insidiously worded, and frankly toxic, now that they joined.
Please don't post comments saying that HN is turning into Reddit. It's a semi-noob illusion, as old as the hills.
https://news.ycombinator.com/newsguidelines.html
I never said that. I said exiled disinformation breathers are polluting HN because they were systematically exiled everywhere else for their dangerous views.
You kind of did though. The website itself hasn't materially changed, so 'HN turning into Reddit' conversation isn't about the features of the application we're using here. It's always about some migration of 'undesirables' into the community.
To be honest, I find your foamy characterizations of the folks you're describing fairly toxic as well.
Isn't it interesting in threads like these that no one ever seems to link to specific examples of their claims?
The commenter may not have quoted you the right rule, but their heart was in the right place. This thread is about building an HN clone, but you've taken it political.
You'll need to _not_ want to do that if you want to continue participating on HN. Otherwise it's a matter of time.
Fully agreed. I've noticed so many political bait comments in the past year or so, and loads of weasel-worded antivax misinformation. Some comments also seem to be trying to push subtle anti-science and anti-left stances, whereas before mostly people just stuck to the technical side of things.
As per usual, the all-trite ruins everything they touch.
I maintain a pretty high quality implementation of Hacker News.
https://github.com/tensorfork/tlarc
It has bcrypt, HN's current CSS, and so on. We use it to manage our TPUs:
http://www.tensorfork.com/tpus
Improvements welcome. The nicest part is how quickly you can iterate on a webpage. The production site pulls from github every 10 seconds, and everything reloads on demand, so I can just push new code to master and see the changes instantly.
(No need to push to github just to see changes locally, of course. Every time you reload, it checks whether any loaded files have changed.)
It's fine as it is, do we really want it to go the direction of Reddit with broken navigation, live chat, typing notifications?
There are several re implementations and browser addons on github. They don't really seem interested, while the user base might be.
Yeah I can think of a thousand reasons why I'd like a fork.
improvements, like, making yourself supreme ruler?
> Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith.
https://news.ycombinator.com/newsguidelines.html
How much does it cost to run a website like HN?
A lot using redis as the backend.
Why would using Redis add significantly to the cost?
Because Redis' database is stored exclusively in RAM, which is an order of magnitude more expensive than even SSD.
Also, long-term maintenance effort and costs would be substantially higher. (Redis persistent-to-disk options, namely AOF and RDB, are notoriously unreliable.) These issues would persist (ha) even with a clustered setup (actually, especially then.)
I've never heard of them being unreliable, and certainly not notoriously so. What sources do you have on this? My own impression formed from reading HN is that opinion around Redis is nearly entirely (almost cultish-ly) positive.
I'm just checking if there is something I missed.
> Because Redis' database is stored exclusively in RAM, which is an order of magnitude more expensive than even SSD.
Maybe, but still very cheap. Unless you are talking about insanely high priced cloud services.
Something like Hetzner would provide you with 128GB of ram for 60 euros / month or 1TB of ram for around 500 euros / month.
Not exactly true. There are things like "Redis on Flash" that redis.com offers for large sets of data. If you enable it, most of your data gets stored in a Flash memory and hence super cheap, in fact even cheaper than things like DynamoDB and stuff.
I don't know how much traffic HN is getting, but i think you should be fine on a 10$ VPS for the most part
2018 is was around 6m requests/day
https://news.ycombinator.com/item?id=26469566
Hardware used at 4m/day was two machines but it runs on one machine
https://news.ycombinator.com/item?id=16076041
So maybe double that now.
(update: I mixed up M5 hosting company with AWS M5 instance types)
Do you have to look at the peaks and design for those?
(The hardware link accounts for this of course, but not sure the hardware scales linearly with average load, because the peaks may not scale linearly with average load)
Related question: is it possible to run HN and serve HN-level traffic on any of the major cloud providers' free tier?
Probably surprisingly little. Also, the publicity HN brings is most likely orders of magnitude more valuable and that makes it much less important to try and optimise the service.
The pages are very light and the algorithms seem to be designed to be quite light, too.
Now, I know neither Arc nor the traffic stats, but I would not be surprised if one or two machines were able to serve entire traffic.
I kinda feel that I should point out that if you want to build a complex app on top of just redid you can as well use zeesql that I develop for exactly this reason.
I should make it open source...
This is interesting, I've seen stuff like RediSQL as well. It would be nice to see this OSS so people can contribute to it!
Ahahah it is the same, I just rebranded RediSQL to ZeeSQL when RedisLabs got the trademark for Redis.