Machine Learning, Finance and Systems Engineering
Vespa is Yahoo’s ‘big data serving engine’ and it was open sourced it in September 2017. We’ve been working on getting it into production deployment for a great start-up called Metsitaba.
Hype is part of most marketing plans, creating a need for products and services - AI hype likes comparing data with oil. Making comparisons with other valuable resources implies great value in drilling for insights in the data. We would agree data is a valuable resource, but the resource it has most in common with isn’t oil.
Many database solutions exist, which one is the ‘best’? Which database vendor/philosophy should you choose? What are the architecture choices we need to understand? This article attempts to explain the underlying mechanisms used in both traditional and alternative technologies.
The human cognitive engine is very robust, meanwhile even insect cognition exceeds that of many ML systems. Some ML solutions can now exceed human ability for specific tasks, but are these algorithms are very fragile. How do we bridge this gap?
Demonstrating a basic Gaussian process fitting of an unknown function, this will be expanded to demonstrate the hyper-parameter search difficulties and issues with cross validation in a temporal setting.
Fear of (Black) swans: Nicholas Naseem Taleb popularised the term Black Swan, referring to extreme outlier events in his book Fooled By Randomness. It originates from John Stewart Mill's 1843 observation on evidence based fallacies. Avoiding them in your business is central to success and fear breeds preparation.
Machine Learning is already able to replace human cognition in low level, high speed, high availabilty tasks, and businesses that recognise the transformational power it brings to their operations will be the future giants.
Tensorflow Data Lake Unstructured Data Vespa.ai Search Data Lake Unstructured Data ML Prerequisites Opinion Database OLTP MMDB ACID MapReduce Hadoop RDBMS Hierarchical Systems Opinion Gaussian Processes Python C++ NLP Ontologies Spark Academic Paper AI Hype Application Startup Incumbent