💾 Archived View for dioskouroi.xyz › thread › 29365530 captured on 2021-11-30 at 20:18:30. Gemini links have been rewritten to link to archived content

View Raw

More Information

-=-=-=-=-=-=-

How to parse EVE Online chat logs (in Common Lisp)

Author: michaelmdresser

Score: 88

Comments: 23

Date: 2021-11-28 02:05:30

Web Link

________________________________________________________________________________

sen wrote at 2021-11-28 08:21:38:

I loved my time playing EVE, spent a fair few years running a major (couple hundred players) Corp in one of the big alliances back pre-pearlabyss. We had a whole suite of web apps that integrated with EVE auth and API, and managed various aspects of the game for us. Threat Intel by putting bots into enemy chats then scraping for keywords (mentions of certain system names), tools to monitor the markets for certain prices/shortages we cared about, custom maps that let players submit scan data, etc etc.

I’ve never experienced any other game that gives you the rush of a major alliance battle in EVE.

It’s an incredible game with a lot of insanely smart people playing and some great communities around it, but regretfully I had to “win” as life is life.

o7

xwolfi wrote at 2021-11-28 08:39:54:

Same but I have to admit it was akin to addiction. Had to stop or I d just do EvE stuff and nothing else :D

sarsway wrote at 2021-11-28 10:41:39:

That's the problem with EvE, a lot of people famously use spreadsheets, and once you start coding your custom tools and get into automating stuff it just never ends. Very soon you're scraping all the markets storing everything in a database looking for profits. Using multiple accounts to scale up your "ISK operation". There's really no limit on what you can do, expect that there's probably a better use of your time :)

ben0x539 wrote at 2021-11-28 08:54:30:

Oh, hey, I had one of these when I was afk-ratting in an Ishtar in Guristas space, those were the days:

https://github.com/ben0x539/eve-log-alert

I'd drop my Ishtar[s] in an anomaly[s], then tab out and probably read hacker news or something, and completely forget about it, usually leading to either dying as I didn't check intel channels or even look for hostiles in the local system, or wasting time as I didn't warp to the next anomaly after clearing the current one. With a simple ruby script, a janky sql query and a pile of ugly heuristics, most of that stopped being a problem, as a very loud noise would remind me to warp away! It hits a "sweet spot" of being totally non-reusable but doing exactly what I need it to while remaining just editable enough to hack in more special cases/heuristics...

It wasn't a super time-efficient way to get space-rich, but at least somewhat effort-efficient, and it netted me enough that I could probably have bought a titan or three. Of course I eventually lost most of my net worth by having it invested in shitty minerals and being unsubscribed as the floor fell out under that market.

Those were the days.

michaelmdresser wrote at 2021-11-28 14:37:44:

I had the exact same motivation as you! I've since moved onto other, much less passive, activities but for a brief moment in time my little tool gave me a lot of peace of mind while fully tabbed out of EVE. Optimizing my game experience for minimal effort/attention and maximum profit has historically taken the enjoyment out of games for me, so I'm glad to have moved on.

amerine wrote at 2021-11-28 04:29:28:

I love love love posts talking about using our skills to make our entertainment ours. Kudos for sharing and much love for sharing how you are making EVE just a little more fun.

michaelmdresser wrote at 2021-11-28 05:22:16:

Thanks for the kind words. One of the things I have always loved about EVE is how open it is (relative to other MMOs) with information like this as well as providing APIs [1] and game data dumps [2]. The data isn't always perfect, and the APIs don't always cover everything the playerbase wants, but I'm just happy any of it exists in the first place! I used the data dump in conjunction with this log parsing to get an automatic "distance from my current location" of enemy players. It isn't anything too special—other developers have made much better tools—but it has been a very fun project.

[1]

https://wiki.eveuniversity.org/EVE_API_Guide

[2]

https://wiki.eveuniversity.org/Static_Data_Export

grimgrin wrote at 2021-11-28 04:36:32:

I like to enhance games with log parsing too. Been playing muds again lately and writing logs with the mud/telnet client (tintin++) to be parsed/spat out in tmux splits, which works great

(as an aside, I started looking and testing mushclient, which is adored by many. nick gammon has been upkeeping it for years and it offers a ton of features/settings)

laserbeam wrote at 2021-11-28 08:50:03:

I used to have an eve chat log parsing tool for getting all the kills in a roam. I'd just ask everyone to x-up once, would then grab their names and timestamps from the chat logs, and then ask zkill what happened to those players during the fleet chat. This was perfect for the kind of chaotic, but small roams we did with Eve Uni, and for the fact that we used to write after action reports after most fleets.

It would sort those kills, and add separators between fights based on time.

It's one of the simplest things I've ever implemented, and I think Eve Uni still used it for years after I stopped playing. Eve chat logs are beautiful.

cschmittiey wrote at 2021-11-29 02:32:51:

Which campus was that at? I feel like I saw something like that used once, but other than that didn’t see it much when I spent my time with E-UNI. I mostly kept to WHC, though.

laserbeam wrote at 2021-11-29 04:25:02:

Mostly low sec and null sec campuses used it. The tool doesn't make sense for whc, it's not a type of space where you roam, having a bunch of small fights in a row.

zeristor wrote at 2021-11-28 12:42:25:

Before this used to be Play By Email games one from the 80s being Empire.

Sean Eddy, he’s probably on HNews, wrote some software to process the emails to get a lead on other teams.

As I remember the story this keen interest helped him hone the skills for genetics analysis, but he could just have been a natural.

https://www.the-scientist.com/uncategorized/a-mind-apart-450...

or:

https://archive.md/3IbBK

I’m reminded of Richard Feynman honing is mathematical skills at school competitions.

Suffice to say these skills with the impetus behind them are hugely useful.

azalemeth wrote at 2021-11-28 15:48:48:

I played EVE for about 4 years around the late 2000s and early 2010s. It's the only game I've ever played where (a) physical units were respected in calculations (e.g. tracking speeds in radians per second; distances in Pc -> Ly -> AU -> km -> m) and (b) other _players_ contained a nonzero number of linux kernel developers...

Yhippa wrote at 2021-11-28 04:44:01:

In general, what is a good language to parse game logs?

stevekemp wrote at 2021-11-28 06:05:24:

Most languages can do most things, so you should pick the one(s) you know best.

That said traditionally perl would be the choice for parsing, and it is still a good candidate. For simple extraction - think "programatic grep" - then awk is also a good choice.

duped wrote at 2021-11-28 06:58:03:

I've found for down and dirty parsing, you can almost always parse simple log/structured output using operations like `split`, `substring`, `prefix`, and `suffix` where :

- `split` splits a string at an index or into substrings separated by a pattern

- `substring` takes a substring of a string for a range of indices

- `prefix` returns true if the start of the string matches a pattern, false otherwise

- `suffix` returns true if the end of the string matches a pattern, false otherwise

Python and C++ both have good standard library functions for these tasks. What's less important than a language for your parser is a language to plug the output into to make something useful.

tomjen3 wrote at 2021-11-28 08:29:43:

Unless the game logs are in json or xml, the easiest language to get started in (assuming a few regexs isn't enough) is whatever language you know, that has a good packrat parser library.

bjoli wrote at 2021-11-28 08:59:40:

Anything with parser combinators or PEG support. I had to learn PEG parsing for a project when I was a kid, and that has been one of my most useful skills.

Yhippa wrote at 2021-11-29 06:30:23:

Pardon, what is PEG?

bjoli wrote at 2021-11-29 10:09:00:

Parsing Expression Grammar.

https://en.m.wikipedia.org/wiki/Parsing_expression_grammar

The examples section under definitions is the easiest way to understand them.

dunefox wrote at 2021-11-28 10:06:58:

I'm using F# and FParsec atm, works great.

2OEH8eoCRo0 wrote at 2021-11-28 05:10:49:

Perl

jhgb wrote at 2021-11-28 09:03:54:

Also known as Practical EVE Re-Engineering Language?