ECDLP, t-sne and job hunting

Heh, I reread that last blog of mine. Boy talk about poorly written! I went ahead and cleaned it up some and it's still kinda bad. This tends to happen when you write via stream of consciousness and then do ad-hoc editing without fully rereading. I'm sure there are writers and editors out there who saw it and went "for the love of all that is sacred re-read what you type!" ah good times. good times. Anyway sorry about that, I'll try and do better.

Anyways, it's been a busy-ish month. I still haven't found a job but I've picked up the pace of applying for them. I wasn't really sure what kind of response I would get. It feels fairly tame so I've compensated and started to up the number applications I'm putting out. It certainly hasn't been like it was 23 years ago! I mention that specifically because I think that was the time I got the most response with hardly any effort. Yeah back in 2000 it was the end of the tech boom and I was still new at my career. At my price point then the demand was kinda crazy. I probably should have asked for more money but alas I was naive to my value. These days each time I go looking for a job it's hit or miss how fast I find something. Something that works well for me I mean, if you take any old job at any old price point you will probably have work the very next day, making next to nothing and be really unhappy doing it. Don't let that happen to you! Regardless I'm still looking.

Beyond job hunting I've spent a couple weeks exploring more stuff with t-sne. I more or less worked through what i was trying to do, which is to say I was trying to make t-sne a tool for data mining. Something beyond visualization and feature reduction. The thing is, sometimes the data is organized in a way that t-sne will auto-group the data for you nicely, but only sometimes. I was trying to write code so that it to adjust itself towards training data automatically. Needless to say it overfit every time. I think the idea is kind of flawed at a certain level. The problem with t-sne (and umap if that's your bag) is they aren't exact. When you do normal data mining you just refine the approximation of the answer over and over again using the training data and cross validation. With t-sne, i can do the same but the output has noise in it and each iteration uses the output with the noise and introduces error.  It treats the noise like facts and they end up dominating any signal in the data. The training data ends up nice and neat and the test data ends up random. In the end I set it aside and called it a failure.

Don't get me wrong it worked really really well at overfitting as if that was some sort of great thing. There just seemed no good way to deal with the noise I was introducing (folding and such wasn't cutting it). I still haven't given up on the idea of doing a 2-d representation of the data and using that to do data mining. It's just i don't feel at this point that t-sne is the way to go. At least not without an 'Ah-ha!' moment where I realize how to stop it from being so biased to the random elements.

I've also been working on the Elliptic Curve Discrete Log Problem (or rather work has continued). It's... gone 'okay'. About 6 months ago I had a small revelation that made it so I could solve a scalar unknown on a prime curve in about 1/2 the steps as before. This built up from an idea I had about 11 months ago. Everything since then has been me trying to push it further (and not doing so). I've learned a lot about some aspects of math I wasn't really well versed in (if at all). I've also done more than a few things that are pretty neat to maybe a small niche crowd but honestly no real progress. I think it's safe to say I'm stuck and just about out of ideas at this point. I say just about cause its hard to let it go. I find myself spending a lot of time thinking and re-thinking certain problems I've come across. Maybe someday I write a blog all about them but for now suffice to say it fills my hours and days when I'm not job hunting.

 

Job hunting and 4 years of exploration

Wow! So, it's weird to think i left my job at Aspire 4 years ago but here I am. I set off from that job wanting to do a number of things. Some of those things had the potential to be money makers. Some were just personal pursuits. In both cases I've had the time to try all of them. It both does and doesn't feel like it's been 4 years. That is to say it feels like the world hasn't changed too much, but I've been busy so those are at odds and maybe it has and I haven noticed. Also, the 2 plus years of pandemic time has a weird separate feeling to it (almost like that time doesn't get counted).

Regardless of how fast or slow it went by I figured it made sense to give a summary of what i've been up to. It's worth throwing out there that I had a blog or two after these last few years which had covered some of it but those are off line now due to the website reset and it's been a good long while since I wrote one any way so I'll give a full run down of all the things I've been up to. It'll at least give you a clear picture of what I did with myself. Here's what I've been up to!

  • Put together a stock investment website ( https://securitiesminer.com/ is still there but i havent' updated it in a long while)
  • Automated financial trading (on https://trade.collective2.com/ I no longer do this though )
  • Automated data mining, including pulling daily data (data from quandl / data.nasdaq.com and the us treasury. turned off currently)
  • Wrote a bunch of add-ons for minecraft (still there on curseforge, a few for java, a bunch for bedrock)
  • Streamed countless hours of gameplay on twitch (Rimworld, Oxygen not included and minecraft)
  • Went on a few trips/vacations (vegas twice for fun, new york for a quandl conference on financial data, Seattle for business)
  • Spent 1000s of hours on algorithms and math around the discrete log problem and the elliptic curve discrete log problem
  • Worked 100s-1000+ of hours  on 3 different games in elecrton.js using babylon.js as the rendering engine (none finished, either lost interest or hit scope problems)
  • Worked out an alternate way to do AI (something I should probably spend more time on. The world's technology could use the variety)
  • Recodified some t-sne stuff I did years ago (clarifying it to be more clear/work better though technically the same stuff i created about 8-9 years ago when t-sne was new)
  • Upgraded computers (it's worth noting this as it's almost always a case of learning where the hardware tech has gone in the last 5-7 years. It's never as easy as "ooh that looks nice")
  • Ran ... lots of running (still do that) no marathons or anything but I still put in probably 4+ miles every other day or so.
  • Played through a number of games I didn't stream: borderlands 3 all the expansions, tiny tina's wonder land, cyberpunk 2077, diablo 4, elden ring (didn't finish , wasn't my thing) and balder's gate 3 (playing that now)
  • Watched countless hours of television, youtube and movies

That's pretty much it, nothing else of note really. I did debate about going to a cryptography conference this last month but ended deciding it was too expensive / made no sense to go right now. Which kind of brings me to where I am today.  That is to say... what's next?

Well I guess it's time to go back to work. That is back to a day job. I had hoped I could produce something truly remarkable with the DLP and ECDLP (discrete logarithm problem and elliptic curve discrete logarithm problem) work I've been doing before I went back. I suppose I have come up with a few remarkable things, but nothing groundbreaking. I figure I need groundbreaking to turn heads. I had hoped I could solve ECCp-131 challenge before I needed to go back to work (https://www.certicom.com/content/certicom/en/the-certicom-ecc-challenge.html) and not just by throwing more cycles at it. That would be quite a feather in my cap and would have opened a lot of doors. It's something i genuinely enjoy working on. So why quit now? the ideas and money are drying up. So I think it's time to turn my focus to getting that job.

It's worth noting if I had managed to do what I wanted my job prospects would probably shift considerably (which I really want as I figured some of those jobs would be a fair bit more interesting). It's hard to stand out to think tanks and places that do novel research without... something. This definitely would have done that. But eh, you can't expect to solve that sort of thing.

I'm currently looking on linked in and am going to try out gun.io . I should probably take a look at the old job sites too. I'm not sure how relevant they are, 20 years ago monster.com was the thing. and 15 years ago dice.com was the up and comer for tech jobs. I'll look in to it some more the next few days.

 

 

The theming is done

Colors have been added! I used the default theme and changed it a wee bit. The default theme is very "IM A DEFAULT THEME". So, you know, i got rid of the offending bits and adjusted a few minor things in the css and site.master. I'm pleased with everything for now. I'll be sure and write a proper blog later.

New blog up and a new blog engine

Hello there! It's been gosh over a year and half since i posted a blog. I've been bad about putting up blogs in the past but this time it wasn't quite my fault. For the really, really long time readers (i mean i've had a blog here for something like a decade... maybe even longer.) You'll remember I used to write custom code for my website/blog. Eventually it was time to start over and I decided to save some time and use some out of the box stuff. After, what? like 6 years of using it, the admin section stopped working. It seems they used some 3rd party hosted file for their libraries (libraries I wasn't familiar with hosted by some unknown company) and since those hosted files went away the admin pages stopped working. I tried the blog briefly but the experience went poorly and as the amount of work for something I barely used escalated I basically went "meh".

So long story made short, new blog and a new engine. It's like i learned nothing! Another out of the box blog engine. I'm sure this will end in tears. (hah) I've got the original blog, but it doesn't play nicely with not being in the website root. And as I didn't write it nor do I have the source code (or the inclination to change it if i did) it's unlikely to show up anytime soon. Might be a good time for a clean break anyway. I'll write some more soon and give updates as to the latest and greatest things going on.

(I'll also zazz up this site.. so white!)