Dim Red Glow

A blog about data mining, games, stocks and adventures.

Vacations and Project 106

Real quick update.... not a whole lot going on right now.I went to aruba and vegas since i  last posted.

the trip to aruba was the worst vacation i've ever been on. why? well while nothing catastrophic happened lots of really crappy stuff did. here's the list. I hurt my back a week before going... making sitting for long times hard and a lot of things impossible. My connecting flight was canceled. I got stuck over night in chicago (despite it being on a new noon when the flight was canceled) which lost me a day of vacation (thanks American Airlines :( ) my next morning flight was at 5:30am. i usually get up around 11am... and was already on 3-4 hour sleep so I was exhausted in a way i shouldn't have been for a leisure vacation. They lose my bags. lesson learned pickup your bag if you have a canceled flight. dont leave it overnight.

My accommodations weren't a resort. This my fault but was a shock when i got there. I thought it was but it was a condo :( This means no comforts or even a gift shop. everything i need is about a mile walk away. Like you know, clothes i didnt have cause of the lost bag (thanks american airlines). There is no bar (wet or otherwise), no place to get clothes/gift shops, no restaurants, no casino, tv was basic cable, no... well anything except a pool and a nearby small public beach. even people didn't exist. the place was so empty. the pools were nice though. There was 1 near by restaurant that did a special 8 course meal thing i did. but otherwise it was a pretty giant waste/not a vacation.

The island is a desert, so not to much to look at (cactus and everything) and it was so windy! I skipped/missed 2 of the 3 excursions i planned because of my back and the canceled flight. My ear got clogged with wax Saturday night right before the 8 course meal (so i had this delightful ringing muffled sound in right ear for the rest of the trip). Turns out there are no pharmacies open in Aruba Sunday. Not that they really could have helped i didn't get it cleaned out till Wednesday at the doctors office. i had allergies Saturday night... and full on sickness Sunday night (flew back Monday). So yeah. all in all it was terrible/felt like work.

Vegas was the exact opposite. i stayed at the Belagio and while it was really nice they did have a few bad things that surprised me. the sink liked to suck air when someone else in the hotel flushed a toilet which means it would go "glug" in the middle of the night. the hall smelled of mold and clearly had a broken pipe right before i got there... so yeah odd for being a fancy hotel.

The pools were amazing as was the food. i saw 1 show (Michael Jackson one). I think i'm just done with musical shows cause it was kinda boring/not my thing. Also I played in the vegas GP which went terribly but they refunded my money. the did this to compensate us for waiting around. They had technical issues so we sat for like 2.5 hours doing nothing (they also gave me a free entry to another gp later that year. i was done with GPs but... free entry so yeah maybe i'll do one more)

I got pretty badly sunburned at the pool. apparently the shirt i wore down took off a lot of my suntan lotion :( let that be a lesson to you all; reapply at the pool. the pool though, amazing! drinks/good times/books/swimming/lots of people. yeah no complaints there.

Finally (unrelated) i've put together another website to track the various things I've been working on. check out project106.com when you get a chance. The site itself is just a living document about the rest of the projects. Many of which I'll mention here eventually.


One Punch Man excercise crazieness

So lately i've been trying to lose weight. Partly cause i like the way i look when i look in the mirror and partly because it's just healthier for me. I had a goal to lose some weight by June 12th butwith the rate weight loss is working out to be. I dont think i'm gonna really come close to my goal.

The main reasons are my age (i'm 41 and metabolism slows down) and my lack of movement most of the day. I run but its not enough. I generally do 4 miles, 3 times a week but my constant sitting doesn't burn a lot of calories so its kind of a wash. Also my muscles seem conditioned to do said 4 mile run quite well without much training. I tried cutting carbs and that helped, but that's not doing much without enough exercise to burn the fat.

So I'm gonna do something that will really, and I mean really, make me lose weight (in a good way). i'm gonna do Saitama's exercise routine (from One Punch Man). I wouldn't do it if it wasn't possible for me. The "hard" part is the running and that's the one part i wont sweat. So what is it?

  • 100 Push-Ups
  • 100 Sit-Ups
  • 100 Squats
  • 10KM Running (that’s 6.2 miles)

every single day.

Okay really? every single day? no. that wouldn't make sense, even when maintaining yourself that seems overkill unless there was some real need. so I'll probably do it 4 days a week. 6.2 miles a day is another 2.2 miles on my normal run. so it will be a little harder at first. the 100 squats and sits ups and push ups wont be bad as long as i break them up in to sets. I'll probably do 2 work outs a day with 3 sets each. so 51 push ups,sit ups and squats  (3 sets of 17) twice a day.

But that's CHEATING! actually, he doesn't really say it was all in one go. in fact i probably should break up the run too. but honestly, i dont really want to cause its going to eat up enough of my day as is. (quite the time sink) And when i get in better shape i'll probably merge the 2 other work outs in to one so it'll be 3 sets of 34 each. but that wont be for a month or so.

am i really that out of shape? this does seem drastic but, its more just something to amuse me. if i absolutely cant find time or hate it, i'll stop. but baring incident, i'll start tomorrow (though my first day i might do 1/2 and really go 100% Wednesday). I'm abandoning my dieting at least till i get going cause not having carbs while exercising is more brutal than I want to be. right now I weigh around 195 I'd really like to be 180. 175 is like my perfect super-in-shape weight that I really havent been in... er ever (well high school, but that was a not yet adult form of me). even when I was in great shape around 28 (13 years ago ugh) i was still like 178. (splitting hairs i know)

Anyway, we'll see how it goes i"ll keep ya all updated. I may or may not post before and after pictures, you'll just have to wait and see hah! really, i'm thinking from the front there will be THAT much difference my face will look a little thinner.  the main thing is I wont have quite the thick core (torso seen from the side) . which is really my goal.

another long over due blog

Hi again faithful readers (you know who you are). So the marathon is back off the plate. Basically, i can't afford to spend the extra cash right now on a trip. Or rather I'm not going to borrow the money for a trip like that. I over spent when i thought the trip was off and upon reflection when it came time to book the trip, it just didn't make sense. So... maybe next year.

So for the last month i've been toying with things with my data mining code base. I removed tons of old code that wasnt being used/tested. i honed it down to just GBM. then i spent some time seeing if i can make a version that boosts accuracy over log loss (normal gbm produces a very balanced approach) i was successful. I did it by taking the output from one gbm and sending it in to another in essence over fitting.

Why would i want to do that? Well there is a contest ( https://www.kaggle.com/c/santander-product-recommendation ) i tried to work on it a little, but the positives are far and few in between. the predictions generally put any given positive at less than 1% chance of being right so i tried bumping the number since right now it returned all negatives. The results are actually running right now. i still expect all negatives but the potential positives should have higher percentages and I can pick a cut off point to round to a positive that is a little higher than .0005% i would have had to use before. All this, so I can send in a result other than all negatives. (which scores you at the bottom of the leader board) Will this give me a better score than all negatives... no idea :)

I tried making a variation on the GBM tree i was using that worked like some stuff i did years ago. it wasnt bad, but still not as good as the current gbm implementation. I also modified the Tree to be able to handle time series data in that it can lock 1 row to another and put the columns in time sensitive order. this allows me to process multi month data really well. It also gave me a place to feed in fake data if i want to stack 1 gbm on top of another i can send in the previous gbm's results as new features to train on (along with the normal training data).

This leads me to where i think the next evolution of this will be. I'm slowly building a multi layer GBM ... which essentially is a form of neural net. The thing i need to work out is how best to sub divide the things each layer should predict. that is i could make it so the GBM makes 2 or 1000 different groups and predicts rows results for each and feeds those in for the next prediction...etc. till we get to the final prediction. the division of the groups is something that can probably be done using a form of multi variable analysis that makes groups out of variables that change together.  figuring out how to divide it in to multiple layers is a different problem all together.

Do you want an AI cause this is how you get AIs! heh, seriously, thats what it turns in to. once you have a program that takes in data builds a great answer in layers solving little problems and assembles them in to a final answer that is super great. well you pretty much have an AI.

Incidently, TSNE might also help here as it I might just feed the tsne results for that layer's training data (fake data included if we are a level or two down) to give the system a better picture of how things group statistically.

In other news, I started using blue apron. This is my first time trying a service like this and so far I'm really enjoying it. I'm pretty bad about going to the grocery store. And going every week ... well that aint gonna happen. This is my way to do that, without doing that :) . I'm sure most people have similar thoughts when they sign up, even if the selling point is supposed to be the dishes you are making. Honestly, I've just been eating too much take out. I don't mind cooking and the dishes they send you to prepare are for the most part really good.



Stopping before I start then starting again

I feel like I've started many things and then abruptly stopped before i really got in to the thick of it. I'd like to share some of the highlights. Doing so will give anyone reading a good picture of where everything is with the stuff i normally (its been over two months) blog about.


Data Mining: Almost 6 months ago I was working on a GBM in a GBM model and there it has sat since then. It's not that I think it's merit-less, it is just that I doubt it'll produce amazing results. So, I'm not really inspired to finish it. Also, I don't have a kaggle contest to work on right now and that helps drive my interest in the algorithm. Truth be told, I'm beginning to think data mining is getting near the end of its "big gains" period. the human analysis part may very well be improvable but that's never interested me much.


Running: I started running, then i stopped, then I started again. to expand on this, it was fine, then i over did it, then I got motivated and eased back in. I've got a marathon in mind I want to go to. It is still 11 weeks away so I'm training up for that. I'll talk more about it when I commit to it fully (basically in about 4 weeks).


Math: I spent a lot of time trying to solve the 3 cubes problem. My most recent attempt sent me down a rabbit hole of general factoring. I actually thought I had a method for doing that, only to realize that my solution to the problem was such a tiny corner case as to be unusable. The net-net is I got nothing I'm pursuing here right now.


Magic: Things continue to wind down. there is a grand prix tourney in Dallas in 2 weeks but I don't think I'm gonna go. I still have fun playing most weeks but I'm definitely not feeling the drive to brew decks like i did. And, lets be honest, i've never felt the competitive spirit this time around. That is to say, I want to win sure, but deck building always took precedence. as it is so much more interesting than playing a known decent deck, which just makes it hard to be competitive.


EM Drive: Have you seen this thing? Its about 2 years old from a "oh hey something new!" perspective. but the last few days I've been reading up on the science and watching youtubes about it. I have to admit I'd like to better understand how it works. I totally get what virtual particles are, but I don't get how they can get the transfer of momentum to them (they are incredibly hard to interact with). Just something I've been messing with and thought I'd include as a bullet point. I doubt I'll build anything in my garage but crazier things have happened.



Update on the updates

First let me share this 'cause you know, ridiculous.

That's pretty much what I'm working on right now in my data mining. it's about half done.

Also I haven't had any marathon training updates. The short version is I've decided to hold off and only give updates when there is something meaningful to share. I never wanted this blog to be work, not to mention I want it to be enjoyable to read. Doing a daily or weekly update for the sake of the update doesn't in my mind fit that. I'm still running, still putting in around 16 miles a week. The weight hasn't really come off. I'm hovering around 197. I figure in a few more weeks if something doesn't give I'll make a more concerted effort to change my diet.

Magic the gathering continues to be a fixture in my life, legacy play especially. I've been contemplating a couple decks in legacy. One I called fraken-mud which was a tool kit mud deck that used birthing pod and eldrazi instead of the usual suspects (like metalworker and forgemaster). It's been inconsistent. Sometimes its really really good, but last week i went 0 for 4... yeah talk about disappointing. I'm also looking at black-white cloudpost deck I want to try that just runs lots of my favorite things (stoneforge, wurmcoil, deathrite, bitter blossom, ugin ..etc) not sure if it'll work or not. I'll try it this friday. I've also got a white painter's deck in the wings I'm gonna try in a few weeks as well. I think going the all red or red white or red blue route on painter grindstone is fine. but I want to try an all white version.

In standard I've been trying to find something interesting to play but eh... it all seems like rehash. I'm probably going to play green white little creatures/good-stuff next time I play. And as for modern, well i have my abzan eldrazi deck still together. I think that's all you get till i get a wild hair and build ... i dunno a new version of affinity. I'd kind of like to try souping up affinity. making it more mid range/stable. but we'll see.