The log of my #100DaysOfCode challenge. Started on [January 8, Tuesday, 2019].
Today's Progress: Worked with downloading datasets from Kaggle using command line and Kaggle API. Downloaded and unziped image files for project using python3.
Thoughts Struggling with command line and getting the path right for the download. Started using GitHub and learnt about convolutional neural networks.
Link(s) to work Starting work on https://www.kaggle.com/c/histopathologic-cancer-detection Getting help for downloading datasets from Kaggle API from https://www.kaggle.com/docs/api#getting-started-installation-&-authentication
Today's Progress: Finally figured out how to add python binaries to PATH and use kaggle API to download datasets! Worked on command line prompts for copying and moving files. Came across healthcare.ai, set up R studio for it. Can't wait to start contributing!
Thoughts Progress feels slow. But progress nevertheless. Have identified and set up system for two projects. Will start working on models from tomorrow.
Link(s) to work https://github.com/aditishenoy/healthcareai-r https://github.com/aditishenoy/pcam
Today's Progress: Started building my first neural network today! Used pcam dataset and basic classification of TensorFlow.
Thoughts Quite excited about exploring a completely new field! Never worked with TensorFlow or image classification before. The large dataset is making my laptop quite slow so that is kind of frustating. But overall, very productive session today!
Link(s) to work https://www.tensorflow.org/tutorials/keras/basic_classification Found this very useful cheatsheet for importing data using python - https://s3.amazonaws.com/assets.datacamp.com/blog_assets/Cheat+Sheets/Importing_Data_Python_Cheat_Sheet.pdf
Today's Progress: Tried finding out where and how to make changes to GitHub repo files.
Thoughts Didn't get much to work today, still trying to figure out how it works.
Today's Progress: Figured out where to make an edit in the repository!
Thoughts Wonder if there is a better way to search for an overview of how all the programs are connected through the models
Link(s) to work http://ugrad.stat.ubc.ca/R/library/e1071/html/naiveBayes.html
Today's Progress: Explored tissue image dataset using openCV and converted into correct format to train model using Python.
Thoughts Found that programming and running two files at the same time for testing if the code works on a smaller dataset would save time instead of testing on large dataset.
Today's Progress: Attempted to train a basic neural network using the image dataset. Started using jupyter notebook.
Thoughts Facing some issues with speed of processing and some run time errors.
Today's Progress: Took a break from the same two projects I have been working on. Learned about Command Line Tools and Bash Scripting!
Thoughts This was a fun exercise today, Codecademy has a great interface for practice!
Link(s) to work https://www.codecademy.com/learn/learn-the-command-line
Today's Progress: Delved into bioinfomatics today. Started Rosalind problems - Bioinformatics Textbook Track
Thoughts Its going to take some time to brush up on basics!
Link(s) to work http://rosalind.info/problems/list-view/?location=bioinformatics-textbook-track https://stepik.org/lesson/2/step/7?after_pass_reset=true&unit=6
Today's Progress: Completed PatternCount function today and started working on FrequentWords function.
Thoughts Getting stuck with - "IndexError: list index out of range". Seems so simple but I'm not getting it. Hoepfully will work it out tomorrow!
Today's Progress: Started working with HTML for the first time today and developed a basic website design!
Thoughts So motivated by today's progress!!
Link(s) to work https://github.com/aditishenoy/aditishenoy.github.io
Today's Progress: Continued working with HTML and CSS for building my personal website.
Thoughts Stuck with choosing an appropriate font. Haha.
Today's Progress: Got some input and changed my course and approach a bit to build personal website.
Thoughts Great that I came across BootStrap CV templates. Would have taken so much more time had I started from scratch.
Today's Progress: Back to Rosalind exercises. Getting stuck at one point, hoping a fresh look at it tomorrow would help!
Thoughts Getting frustrated at this point.
Today's Progress: Personal Website finally taking some shape. Should be done with it soon!
Thoughts Very happy with progress today!
Today's Progress: Completed challenges in 1.2 Hidden Messages in the Replication Origin from Rosalind Bioinformatics Textbook Track and started with 1.3 today
Thoughts Fresh outlook towards the same problem made it easier to solve!
Today's Progress: Completed challenges in 1.3 Some Hidden Messages are More Surprising than Others from Rosalind Bioinformatics Textbook Track!
Thoughts Sometimes these exercises are fun!
Today's Progress: Dabbled a little bit in differnt projects today.
Thoughts Didn't feel like today was particularly productive. Maybe not a good idea to keep this one hour of coding at the end of a long day.
Today's Progress: Worked with some WebDev. Trying to add a reference to a pdf on the desktop.
Thoughts NEED to wind up with this website by tomorrow. Its a week since I started this project.
Today's Progress: Done with website! Woohoo!
Thoughts Feeling happy about having completed one project since I started this challenge.
Link(s) to work https://aditishenoy.github.io/
Today's Progress: Final touches to website and making sure everything works.
Thoughts Trying to figure out how to add pdf
Today's Progress: All final touches done!
Thoughts Nervous to publish. Hoping for some good feedback!
Today's Progress: Done with 3 Rosalind Bioinformatics Stronghold problems today!
Thoughts Improved my speed of solving these problems today. I remember starting the first problem 4 years ago and gave up. And today I solved it within 10 minutes. Happy to see how far I've come!
Link(s) to work http://rosalind.info/problems/list-view/
Today's Progress: Tried fixing bugs on my website.
Thoughts Too tired to do anything more intense.
Today's Progress: Worked on 4th problem of Rosalind Bioinformatics Stronghold.
Thoughts Getting stuck and its frustrating. Hopefully, I'll crack it tomorrow.
Today's Progress: Worked with some Rosalind problems.
Thoughts --
Today's Progress: Started with Bioinformatics Contest: February 2019 today! (1/8)
Thoughts Started working with Problem 3. Transposable Elements. Seems challenging but I am going to try my best and will see if I make it through :P
Link(s) to work https://stepik.org/lesson/202328/step/2?unit=179664
Today's Progress: Worked with Bioinformatics Contest (2/8) and did a rosalind problem.
Thoughts Finding the contest challenging.
Today's Progress: Training a neural network for heart disease prediction and continued working on yesterday's problem.
Thoughts Too tired today to do productive work.
Today's Progress: Was figuring out some things with github and restarted some work with pcam.
Today's Progress: Worked more with pcam dataset and keras today. Trying to get a hang of using .h5 files.
Thoughts Using HDF5 is a really interesting alternative to what I'm used to (numpy/pandas) dataframes.
Link(s) to work https://github.com/aditishenoy/Training/tree/master/pcam
Today's Progress: I focused mostly on my thesis project today. Worked with command line and Linux containers today. Struggled with stand-alone Cuda so using tensorflow GPU instead.
Today's Progress: Tried fixing a bug in the container I started working on yesterday.
Thoughts Getting stuck with some version compatibility issues.
Today's Progress: After almost a week of struggling to get the code to work, I finally fixed the bugs and succesfully got it to work on the GPU today.
Thoughts Despite the frustration faced while programming, the happiness when the desired outcome is reached is indescribable. I AM SO SO HAPPY.
Today's Progress: Worked with Keras and started breaking down concepts into small understandable parts in my Training repo on github
Thoughts Still trying to figure out git and commits
Today's Progress: Started with W1 of CNN course - really helps with understanding and applying the concepts for my project!
Today's Progress: Continued working with this today - Worked with Keras and started breaking down concepts into small understandable parts in my Training repo on github
Today's Progress: Spent time fixing some bugs in the code today. Can't figure out where the problem is
Today's Progress: Tested some changes to a deep CNN on CIFAR10 dataset
Thoughts So excited to learn more about NN!
Link(s) to work https://github.com/keras-team/keras/blob/master/examples/cifar10_cnn.py
Today's Progress: Worked with some parameter tuning today.
Thoughts Don't think I've ever stuck to something for 40 days in a row before!
Today's Progress: Trying to set up a container on my machine
Thoughts Super fascinated by the concept of containers
Today's Progress: Set up docker on my machine. Trying to access python from the container now.
Today's Progress: Found singularity and vagrant easier to install and navigate through.
Thoughts Getting stuck with accessing the files on the host machine through the guest. Any ideas on how to do this?
Link(s) to work Using this: https://www.vagrantup.com/docs/synced-folders/basic_usage.html Changed the first parameter to the file path on host and second parameter to path for guest.
Today's Progress: Figuring out how to get a probability distribution using softmax.
Thoughts Getting frustrated cause I understand the concept but is getting challenging to implement
Today's Progress: Continued working on this and figured it out today! The labels need to be converted to categorical first to change it from a regression to classification problem.
Today's Progress: Fixed some inconsistencies in this code today. Finally producing a non-nan output!
Today's Progress: Worked with loss functions today!
Today's Progress: Getting the class probabilities from the softmax layer as an output. Still working on this and plotting it as a histogram. Worked on documenting some parts of my codes as well.
Thoughts I felt too distracted today
Today's Progress: Starting a new project today - training cifar10 dataset using different nn architectures. Started off with a basic sequential model. Will optimize and move on to U-net tomorrow.
Link(s) to work https://github.com/aditishenoy/Thesis--Neural-networks
Today's Progress: Started learning and using vim today! Love it!
Today's Progress: Working with creating visualization of Actual vs Predicted values
Today's Progress: Working more with creating visualization of Actual vs Predicted values
Today's Progress: Trained the cifar10 dataset with and without data augmentation on GPU
Today's Progress: Attempting to train cifar10 using u-net
Today's Progress: Continued with figuring out u-net
Today's Progress: Getting a dimensionality mismatch while fitting the model using unet. Hopefully will figure this out during tomorrow's coding session!
Today's Progress: Worked with performance metrics
Today's Progress: Deployed mean absolute error and relative absolute error for the predictions and visualised them. Anyone have tips for better visualisations in python other than matplotlib line plots?
Today's Progress: Found an error in the predictions so worked on getting those right today
Today's Progress: Everything that was working last week stopped working suddenly. Spent today trying to understand what happened 😢
Today's Progress: Tried going to the concepts on understanding how to convert regression problem to classification to break down the problem step by step.
Today's Progress:Successfully made the bins and got the categorical cross entropy to work. Getting loss as nan for some reason. 🤔 Any suggestions on what I can do to fix this?
Today's Progress: Changed the loss function from categorical cross entropy to MSE and the curse of the nan's stopped! 🎉 Training speed is significantly slower though. Need to still figure out how to get predictions as class probabilities.
Today's Progress:Learnt about this beautiful resource @TensorBoard. Produces such beautiful visualisations to help figure out what's wrong with my network! 🎉
Today's Progress:continued with trying to make this work 🤦
Today's Progress Code stopped giving nan loss!!! 🎉🎉 Just need to figure out how to calculate and sort the predictions now!
Today's Progress Obtained class probabilities and figured out how to calcuate cummulative sum from softmax function!
Today's Progress Wasn't too productive today 😓 Getting stuck again with training CIFAR10 using U-Net architecture.
Today's Progress Worked with performance metrics based on new predictions
Today's Progress Can't believe its already Day 70!! Worked with figuring out the dimentionality mismatch for unet architecture today.
Today's Progress: Took a break from neural networks and worked with some HTML for my website today.
Today's Progress: Trying to find out why my nn is overfitting. Didn't do much at all.
Today's Progress: Continued with HTML today. Its refreshing to change projects to something completely different sometimes!
Today's Progress: Changing hyper-parameters to optimize nn and stop overfitting
Stopped updating here from D74 - D100.