Capstone Projects

GenAI Sales: “Cognitive Selling”

As part of ongoing consultation to one of Silicon Valley’s leading GTM consulting groups, we developed a set of GenAI techniques to augment enterprise solution selling (like MEDDIC). The work was a mix of conceptual approaches with technical feasibility assignments to explore how to ensure causal explainability in the models to avoid negative consequences of hallucinations. We also concluded that a Mixture of Experts approach was required to automatically select from a range of fine-tuned and PE models to best suit the sales context. This is particularly critical to avoid misalignment — e.g. competitive intelligence data should not be confused for sales playbook data etc. 

GenAI News Research Tool

For one of the largest media groups in the GCC region, we explored several GenAI use cases, both within digital transformation and innovation realms. A major PoC is the development of a journalist’s news research tool using GenAI to greatly accelerate the research and story pipeline process. The use of LLMs, agents, guardrails is proposed to be combined with novel research in fact-checking techniques using mostly novel prompt-engineering schemas. We also architected a “CitizenGPT” open journalism platform that has the novel feature of radically open ranking and reputation algorithms for the sake of journalistic transparency.

GenAI Online Travel Agency

For a client in the travel industry, we evaluated a number of GenAI strategies and technical vectors (LLMs, agents, dense embedding search) to facilitate various online travel use cases. A major challenge was semantic mapping to compressed lexical APIs. A novel aspect was a unique prompt engineering and fine-tuning strategy required to reach exceptional consumer-friendly performance for itinerary management, which involved a number of unique problems. We also benchmarked open source smaller model implementations against GPT-4 etc.

GenAI Digital Therapeutics

For a digital therapeutics start-up, we developed a PoC for showing how an entire catalog of holistic therapeutic interventions could be delivered using GenAI extractive summarization (e.g. using dense vector search and LLMs). The novel factor was the use of long-term memory to ensure coherent therapeutic responses aligned with patient history. An additional and challenging novel factor was the use of digital behavioral frameworks (COM-B model e.g. gamification) custom generated for the target patient and therapy.

GenAI Semantic (Neural) Search Reference Architecture

For a UK start-up in the “Trusted Reviews” space, we developed a semantic search reference architecture using: Apache Solr, Sentence Transformers, Bi-Encoders, Cross-Encoders, Learn-To-Rank (LTR) models with semantic enhancement, Semantic-Graph Traversal, Semantic Query-classifier, QA Transformers etc. Some elements localized to MS Azure (e.g. Synapse) because the client was Azure-centric. We conducted side-by-side solution comparison to AWS native services (e.g. OpenSearch).

Digital Experience Platform (DXP) AI Strategy and Research (GenAI)

For Sitecore, a sales-focussed DXP vendor with >5000 clients, including many of the world’s leading brands (Revlon, Volvo, Microsoft) we developed an “AI-first” tech/product roadmap with a particular focus on intersections between composable headless CMS and Metaverse (immersive commerce, or 3D stores). Included research of novel “prompt to site” approach to web authoring using Transformers.

Web3 Metaverse Platform

For a Bay Area start-up, we helped define a set of Web3 protocols and dApps for the implementation of a hybrid Metaverse-Web3 data platform for building customer experiences that can exist in the emerging “Decentralized Metaverse”. This is based upon my experience with cryptographic techniques (for which we have patents).

AI-Powered Data Fabric

For a stealth start-up out of UC Berkeley’s incubator program, we led architecture for a Data Fabric solution using MMOG gaming (back-end) technology that is “Digital Twin” compatible via the novel use of the highly composable Entity-Component-System pattern.

The technical approach that was novel was to build a Data Fabric architecture easily amenable to natural language query. The work involved NLP, Elasticsearch and novel AI solutions related to knowledge management, including graph processing. The architecture was AWS “serverless”.

Deep AI Personalization for Fintech Customer Acquisition

For Prosper Inc, we built a series of PoCs and filed related patent applications in the area of user-journey personalization. This included novel AI multimodal embeddings and a variety of related techniques. The guiding principle was to develop tools that marketing folks could “self-serve” to assist with automating marketing decisions across the entire user journey without relying upon data scientists and bespoke analyses.

The work including handling “AI Alignment/Safety/Bias” to mitigate “disparate impact” wherein users must not be discriminated against based upon parameters that are illegal (e.g. gender, race). These factors are easily masked in certain types of AI approach, so we developed a framework for “explainable AI” using novel “adversarial” techniques.

Attempts were made to model “financial personality” via correlation of financial activity with psychometric profiles. This work was patented.

AI Chat Bot – “Finance Speak”

We built an AI bot that could speak “consumer finance”, especially lending related. The goal was to provide a novel technical capability for the client in order to pivot towards a more horizontal market (e.g. financial wellness). Calling an agent is too much friction whereas chatting to a regular “support/faq bot” is too frustrating due to their typical lack of sophistication.

Much of the patented work was attempting a so-called Level-3/4 bot with conversational capabilities via a “financial-speak language model” capable of handling dialogs that strayed from the happy path by attempting to track user attention. Techniques included a novel application of embeddings in a conjoined vector space that attempted to model typical finance-speak pathways. A blend of supervised and unsupervised techniques were used to improve the handling of ambiguity in customer intent.

Blockchain: Retail Lineage

Combining previous work to develop blockchain protocols for lending with prior experience in e-commerce, we designed a platform to tackle fraud in the multi-billion luxury goods market.

There are two types of fraud – blatant knock-offs and deceitful trading of goods claiming them to be originals. The latter has led to specialized websites like Flight Club who act as an intermediary to vouch for authenticity of branded luxury goods.

We developed on-chain protocols to produce transferable point-of-sale digital receipts via what are called NFTs. The project was framed as an ecosystem approach whereby consumers and merchants could trade goods with digital receipts. We spoke with various luxury brands, such as Yeezy, Burberry and Louis Vitton. 

ML: Dynamic Demand Pricing

We provided technical consultation for the Credit Risk division of Prosper Inc. to explore the mathematical and computational foundations of elastic pricing (see Paul’s slides) for unsecured loans based upon the seminal work of world-leading scientist Dr. Robert Philips.

Pricing of loans by lenders is typically a function of credit risk, but this can be extended to be a function of demand elasticity and competitor pricing.

Our job was to interpret the foundations of pricing credit products using dynamic methods to the point that our credit team had a basis for a product. Most of my contribution was in parsing the dense literature and interpreting some of the key mathematical principles into Python to demonstrate how to map the methods to an on-line architecture. The PoC was adapted using MLOps methods to achieve dynamic releases of the algorithms.

Computer Vision: Financial Document Verification

For Prosper Inc, we explored the bottleneck problem of financial document verification for online lending.

Using a novel ensemble of computer vision techniques, such as Microsoft Research’s stroke-width-transform, we built a containerized patented pipeline in Numpy that, unlike conventional “check deposit” systems, could analyze financial documents at any angle and with any degree of background noise, such as a user might casually snap with their mobile camera (unconstrained due to the lack of a mobile app). Under such conditions, my solution out-performed leading finance-OCR solutions by Lead Tools and DL solutions using Google Cloud Vision. The end result was an easier and quicker application process for customers with overall conversion increase.

Blockchain: Lending IaaS

Prosper Inc. is a fintech leader who pioneered marketplace lending, connecting individual lenders with borrowers. Using digital orchestration and ML-based credit rating, unsecured loans are closed far quicker than banks.

But what’s next? Running an innovation lab, we led development of a suite of patented blockchain protocols that would allow frictionless instant lending, and even pre-emptive lending via “smart money” (read my article on Medium). This is more than just publishing loans on chain. That would be like storing photos of dollar bills and calling it “Bitcoin”. Hardly!

The project was part of an engagement to explore Horizon 3 futures and IP generation as a future-proof vehicle for growth via category-defining user experiences (instant “smart” loans).

AI: Perceptual Color Harmony and Natural Language Generation (NLG)

Color sits at the center of any decor project. However, the computational, biological and perceptional reality of color are very different. Color perception is highly influenced by saliency, or what observers find interesting in a picture, even more so within a decor context. The perception of art is deeply mysterious — it is possible to hang a picture in our lounge, glance at it a hundreds of times and yet fail to notice its true (veridical) colors.

Working at Art.com for the CTO, we founded an innovation lab with significant risk-investment from the CEO. We led development of some of the world’s best (and patented) color-perception technology to provide a means for Art.com to pivot into a wider decor market with novel customer-satisfying solutions. We built Natural Language Generation (NLG) methods (pre LLMs) that could describe the colors and style of any piece of art using the vernacular of an interior designer (using a language model derived from interior design blogs and a novel “art style” classification method). 

Hardware/IoT: Digital Art

After extensive analysis of how to solve the Discovery problem in art and decor, we helped create a skunkworks project to build a novel digital art display that could adapt to user tastes and interests over time. This exploited my early career as a hardware engineer and chip designer.

We created a hardware lab (metaphorically and physically) to create a hardware device, embedded software and a mobile app at lightning speed with tight budgets. We also created (and patented) a novel file player format (called .art) to support long-playing artworks that evolve over periods of up to 1 year. And, to bootstrap the art platform, we worked with the marketing team to create an extensive network of early adopter artists.

The pilot was a success with excess demand for devices and a number of key patents. The technology was acquired by Walmart.

E-commerce Digital AI Lab

Discovery of art and decor (or any aesthetic product) is complicated by the tendency of users to say “I know what I like when I see it“. We led comprehensive design research programs using AI, neuroscience, neurobiology, psychology and ethnography to find foundational (and computational) methods that could predict what a user might like upon seeing it. [Image to the right shows tastes in art as discovered via social curation networks in the data.]

The work was done in a “innovation lab” that we set up. Some of the concepts are documented in this slide share.

We collaborated with world’s experts in color at the Pantone Color Institute and many leading decor brands.

Using UX research, we invented a method to discover the theoretical limits of how quickly a user could “like it when they see it”. This was an extension of perceptional vision work at MIT.

The work generated substantial IP patent portfolio for the client that significantly impacted valuation multiples at the time.

Causal/Graph AI: Consumer Prediction

Sixth Sense was a start-up that our principal, Paul Golding, co-founded in Cupertino, CA with a stellar team lead by Silicon Valley luminary Bobby Srinivasan. Its goal was to predict consumer “next action” or “in the moment” behaviors as close to real-time as possible using mobile network, supplemental location and financial transaction data (from the Visa network).

Paul’s contribution was deep technical product design that proposed a novel method called Knowledge Signature Analysis (now more commonly called Causal AI or Knowledge-Graph CNN techniques).

IoT/ML/AI: Sensor Networks

McLaren is a tour de force in the world of high-precision F1 racing and luxury sports cars. McLaren’s brand is synonymous with “performance engineering”.

We were hired by McLaren’s Applied Technologies to extend the reach of “performance engineering” to new clients (and revenue streams) by architecting a sensing “stream-processing” platform that could ingest data from any number of IoT devices and process the data in real-time using ML, AI or functional programming.

The work exploited McLaren’s reputation with real-time sensing of F1 racing cars. The goal was to  attract clients in the “performance sports” industry (e.g. Nike, Specialized) to build new and sustainable revenue streams with low volatility.

The platform and vision were central to the initial growth and reputation of the MAT division.

O2 UK Labs: Building at the “speed of thought”

Our principal, Paul Golding, was the interim technical co-founder as part of an agile cross-functional team — an “innovation lab” — inside of O2 (UK) and Telefonica R&D (Spain). During his tenure, he formulated and executed the idea of “Operator 2.0” — the translation of Web 2.0 methods, including product-centric methods (Lean/Agile) to carrier ops. Projects included:

#Blue – a text-messaging API built using “lean” methods and modern software techniques at 1/10th the cost of historical carrier product costs: one of the earliest applications of “Lean” inside of a large org (use case shared with Eric Ries.)

O2 Incubator – the first start-up incubator created by a telco in Europe, sponsored by Sir Simon Devonshire who went on to run Telefonica’s Wayra London. It was part of an “innovation network” strategy.

connFu — see next project.

During this engagement, Paul evangelized the concept of Connected Services and wrote a book to explain it (read review here). It has been widely read by carrier executives globally.

cPaaS Innovation: connFu

Whilst consulting for O2 UK, Paul approached the venture fund of the R&D division of Telefonica to fund a radical new approach to building carrier services: connFu.

This was both a platform and a scripting language (“Domain Specific Language” – DSL) — see example in adjacent image. Paul led the design of the scripting language, based upon Ruby given its popularity at the time (Ruby on Rails).

connFu was a developer-first product designed to enabled developers (internal/external) to create telco services “at the speed of thought” versus the old carrier waterfall method that hindered innovation on telco platforms.

The telephony services were powered by a telephony server product (Voxeo) atop of the Jajah IP-telephony network (strategically acquired by Telefonica). The lab included an extension into Silicon Valley to “infuse” the telco with modern techniques, such as Lean, Agile and TDD.

This was the first – and seemingly only – time that a carrier built its own developer-friendly scripting language and fully embraced the APIs and Open Platforms approach. This work led to Paul writing the book “Connected Services” to explain how to understand and map open platform methodologies to telcos.

Motorola Labs: Mobile Mash-Ups

Motorola hired our principal, Paul Golding, as a technical “co-intrapreneur” to build a brand new business unit that offered scalable apps services to carriers. He assumed the interim role of Global Chief Architect, bringing “Web 2.0” methods into a more traditional J2EE world.

Besides providing the entire apps strategy, Paul was hands-on with architecting numerous solutions, including content management systems, mobile TV platforms, voice services, push-to-talk and music streaming using SIP signaling extensions.

Paul also started and ran a “Mash-Ups Lab” in Chicago for building service mash-ups atop of the platform. We built the world’s first location-based taxi-hailing system that also included single-click voice connections (using IMS) between driver and passenger. It was Uber before Uber.

Paul provided IMS consulting expertise to the Microsoft Exchange team as they started to embrace IP-telephony services and also consulted for the GSMA to benchmark proposed IMS chat protocols against existing protocols like Jabber etc.