Feed aggregator

1. Data Organization in Spreadsheets — Focusing on the data entry and storage aspects, this article offers practical recommendations for organizing spreadsheet data to reduce errors and ease later analyses. The basic principles are: be consistent, write dates like YYYY-MM-DD, do not leave any cells empty, put just one thing in a cell, organize the data as a single rectangle (with subjects as rows and variables as columns, and with a single header row), create a data dictionary, do not include calculations in the raw data files, do not use font color or highlighting as data, choose good names for things, make backups, use data validation to avoid data entry errors, and save the data in plain text files. A “must-read” for anyone who works with data. (via Thomas Lumley)
2. Toward an API for the Real Numbers — To our knowledge, this is the first exploration of a practical general purpose real number type that both reflects the mathematical laws of the real numbers, and also supports exact comparisons in situations in which that’s normally expected. (via Morning Paper) (via Tim Bray)
3. Sensors Printed Directly Onto Skin — Here, we report a universal fabrication scheme to enable printing and room-temperature sintering of the metal nanoparticle on paper/fabric for FPCBs and directly on the human skin for on-body sensors with a novel sintering aid layer. Consisting of polyvinyl alcohol (PVA) paste and nanoadditives in the water, the sintering aid layer reduces the sintering temperature. Together with the significantly decreased surface roughness, it allows for the integration of a submicron-thick conductive pattern with enhanced electromechanical performance. Various on-body sensors integrated with an FPCB to detect health conditions illustrate a system-level example. (paywalled paper)
4. A Quarter Century of Hype – 25 Years of the Gartner Hype Cycle — A presentation of several novel ways to visualize 25 years of the Gartner Hype Cycle. The good stuff starts about 1m40s in.

The field of AI product management continues to gain momentum. As the AI product management role advances in maturity, more and more information and advice has become available. Our previous articles in this series introduce our own take on AI product management, discuss the skills that AI product managers need, and detail how to bring an AI product to market.

One area that has received less attention is the role of an AI product manager after the product is deployed. In traditional software engineering, precedent has been established for the transition of responsibility from development teams to maintenance, user operations, and site reliability teams. New features in an existing product often follow a similar progression. For traditional software, the domain knowledge and skills required to develop new features differ from those necessary to ensure that the product works as intended. Because product development and product operations are distinct, it’s logical for different teams and processes to be responsible for them.

In contrast, many production AI systems rely on feedback loops that require the same technical skills used during initial development. Similarly, in “Building Machine Learning Powered Applications: Going from Idea to Product,” Emmanuel Ameisen states: “Indeed, exposing a model to users in production comes with a set of challenges that mirrors the ones that come with debugging a model.”

As a result, at the stage when product managers for other types of products might shift to developing new features (or to other projects altogether), an AI product manager and the rest of the original development team should remain heavily involved. One reason for this is to tackle the (likely) lengthy backlog of ML/AI model improvements that will be discovered after the product engages with the real world. Another, of course, is to ensure that the product functions as expected and desired over time. We describe the final responsibility of the AI PM as coordinating with the engineering, infrastructure, and site reliability teams to ensure all shipped features can be supported at scale.

This article offers our perspective into the practical details of the AI PM’s responsibilities in the latter parts of the AI product cycle, as well as some insight into best practices in execution of those responsibilities.

Debugging AI Products

In Bringing an AI Product to Market, we distinguished the debugging phase of product development from pre-deployment evaluation and testing. This distinction assumes a slightly different definition of debugging than is often used in software development. We define debugging as the process of using logging and monitoring tools to detect and resolve the inevitable problems that show up in a production environment.

Emmanuel Ameisen again offers a useful framework for defining errors in AI/ML applications: “…three areas in particular are most important to verify: inputs to a pipeline, the confidence of a model and the outputs it produces.” To support verification in these areas, a product manager must first ensure that the AI system is capable of reporting back to the product team about its performance and usefulness over time.  This may manifest in several ways, including the collection of explicit user feedback or comments via channels outside of the product team, and the provision of mechanisms to dispute the output of the AI system where applicable. Proper AI product monitoring is essential to this outcome.

I/O validation

From a technical perspective, it is entirely possible for ML systems to function on wildly different data. For example, you can ask an ML model to make an inference on data taken from a distribution very different from what it was trained on—but that, of course, results in unpredictable and often undesired performance. Therefore, deployed AI products should include validation steps to ensure that model inputs and outputs are within generally expected limits, before a model training or inference task is accepted as successful.

Ideally, AI PMs would steer development teams to incorporate I/O validation into the initial build of the production system, along with the instrumentation needed to monitor model accuracy and other technical performance metrics. But in practice, it is common for model I/O validation steps to be added later, when scaling an AI product. Therefore, the PM should consider the team that will reconvene whenever it is necessary to build out or modify product features that:

• ensure that inputs are present and complete,
• establish that inputs are from a realistic (expected) distribution of the data,
• and trigger alarms, model retraining, or shutdowns (when necessary).

The composition of these teams will vary between companies and products, but a typical cross-functional team would likely include representatives from Data Science (for product-level experimentation and inference task validation), Applied Science (for model performance and evaluation), ML Engineering (for data and feature engineering, as well as model pipeline support) and Software/Feature Engineering (for integration with the full stack of the AI product—such as UI/UX, cloud services, and dev ops tools). Working together, this post-production development team should embrace continuous delivery principles, and prioritize the integration of any additional necessary instrumentation that was not already implemented during the model development process.

Finally, the AI PM must work with production engineering teams to design and implement the alerting and remediation framework. Considerations include where to set thresholds for each persona, alert frequency, and the degree of remediation automation (both what’s possible and desired).

Inference Task Speed and SLOs

During testing and evaluation, application performance is important, but not critical to success. In the production environment, when the outputs of an ML model are often a central (yet hidden) component of a greater application, speed and reliability are critically important. It is entirely possible for an AI product’s output to be absolutely correct from the perspective of accuracy and data quality, but too slow to be even remotely useful. Consider the case of autonomous vehicles: if the outputs from even one of the many critical ML models that comprise the vehicle’s AI-powered “vision” are delivered after a crash, who cares if they were correct?

In engineering for production, AI PMs must take into account the speed at which information from ML/AI models must be delivered (to validation tasks, to other systems in the product, and to users). Technologies and techniques—such as engineering specifically for GPU/TPU performance and caching—are important tools in the deployment process, but they are also additional components that can fail, and thus be responsible for the failure of an AI product’s core functionality. An AI PM’s responsibility is to ensure that the development team implements proper checks prior to release, and—in the case of failure—to support the incident response teams, until they are proficient in resolving issues independently.

AI product managers must also consider availability: the degree to which the service that an AI product provides is available to other systems and users. Service Level Objectives (SLOs) provide a useful framework for encapsulating this kind of decision. In an incident management blog post, Atlassian defines SLOs as: “the individual promises you’re making to that customer… SLOs are what set customer expectations and tell IT and DevOps teams what goals they need to hit and measure themselves against. SLOs can be useful for both paid and unpaid accounts, as well as internal and external customers.”

Service Level Indicators, Objectives, and Agreements (SLIs, SLOs, and SLAs) are well-known, frequently used, and well-documented tools for defining the availability of digital services.  For cloud infrastructure some of the most common SLO types are concerned with availability, reliability and scalability. For AI products, these same concepts must be expanded to cover not just infrastructure, but also data and the system’s overall performance at a given task. While useful, these constructs are not beyond criticism. Chief among the challenges are: choosing the correct metrics to begin with, measuring and reporting once metrics are selected, and the lack of incentive for a service provider to update the service’s capabilities (which leads to outdated expectations). Despite these concerns, service level frameworks can be quite useful, and should be in the AI PM’s toolkit when designing the kind of experience that an AI product should provide.

Durability

You must also take durability into account when building a post-production product plan. Even if well-designed, multi-layer fault detection and model retraining systems are carefully planned and implemented, every AI-powered system must be robust to the ever-changing and naturally stochastic environment that we (humans) all live in. Product managers should assume that any probabilistic component of an AI product will break at some point. A good AI product will be able to self-detect and alert experts upon such a failure; a great AI product will be able to detect the most common problems and adjust itself automatically—without significant interruption of services for users, or high-touch intervention by human experts.

There are many ways to improve AI product durability, including:

• Time-based model retraining: retraining all core models periodically, regardless of performance.
• Continuous retraining: a data-driven approach that employs constant monitoring of the model’s key performance indicators and data quality thresholds.

It’s worth noting that model durability and retraining can raise legal and policy issues. For example, in many regulated industries, changing any core functionality of an AI system’s decision-making capability (i.e., objective functions, major changes to hyperparameters, etc.) require not only disclosure, but also monitored testing.  As such, an AI Product Manager’s responsibility here extends to releasing not only a usable product, but one that can be ethically and legally consumed. It’s also important to remember that no matter what the approach to developing and maintaining a highly durable AI system, the product team must have access to high quality, relevant metrics on both model performance and functionality.

Monitoring

Proper monitoring (and the software instrumentation necessary to perform it) is essential to the success of an AI product. However, monitoring is a loaded term. The reasons for monitoring AI systems are often conflated, as are the different types of monitoring and alerting provided by off-the-shelf tools. Emmanuel Ameisen once again provides a useful and concise definition of model monitoring as a way to “track the health of a system. For models, this means monitoring their performance and the equity of their predictions.”

The simplest case of model monitoring is to compute key performance metrics (related to both model fit and inference accuracy) regularly. These metrics can be combined with human-determined thresholds and automated alerting systems to inform when a model has “drifted” beyond normal operating parameters. While ML monitoring is a relatively new product area, standalone commercial products (including Fiddler and superwise.ai) are available, and monitoring tools are incorporated into all the major machine learning platforms.

Separate from monitoring for model freshness, Ameisen also mentions the need to apply technical domain experience in designing monitoring systems that detect fraud, abuse, and attack from external actors. AI PMs should consult with Trust & Safety and Security teams to combine the best principles and technical solutions with existing AI product functionality. In some specific domains—such as financial services or medicine—no easy technical solutions exist. In this case, it is the responsibility of the AI product team to build tools to detect and mitigate fraud and abuse in the system.

As we’ve mentioned previously, it’s not enough to simply monitor an AI system’s performance characteristics. It is even more important to consistently ensure that the AI product’s user-facing and business purposes are being fulfilled. This responsibility is shared by the development team with Design, UX Research, SRE, Legal, PR, and Customer Support teams. The AI PM’s responsibility is again to orchestrate reasonable and easily repeatable mitigations to any problems. It is crucial to design and implement specific alerting capabilities for these functions and teams. If you simply wait for complaints, they will arise far too late in the cycle for your team to react properly.

No matter how well you research, design, and test an AI system, once it is released, people are going to complain about it. Some of those complaints will likely have merit, and responsible stewardship of AI products requires that users are given the ability to disagree with the system’s outputs and escalate issues to the product team.

It is also entirely possible for this feedback to show you that the system is underserving a particular segment of the population, and that you may need a portfolio of models to serve more of the user base. As an AI PM, you have the responsibility to build a safe product for everyone in the population who might use it. This includes consideration of the complexities that come into play with intersectionality. For example, an AI product might produce great outcomes for wealthy, American, cisgender, heterosexual, White women—and although it might be tempting to assume those outcomes would apply to all women, such an assumption would be incorrect. Returning to previous anti-bias and AI transparency tools such as Model Cards for Model Reporting (Timnit Gebru, et al.) is a great option at this point. It is important not to pass this development task off to researchers or engineers alone; it is an integral part of the AI product cycle.

If done right, users will never be aware of all the product monitoring and alerting that is in place, but don’t let that trick you. It’s essential to success.

Post-Deployment Frameworks

One question that an AI PM might ask when pondering these post-production requirements is: “This seems hard; can’t I just buy these capabilities from someone else?” This is a fair question, but—as with all things related to machine learning and artificial intelligence—the answer is far from a binary yes or no.

There are many tools available to help with this process, from traditional vendors and bleeding edge startups alike. Deciding what investment to make in MLOps tooling is an inherently complex task. However, careful consideration and proactive actions often lead to defendable competitive advantages over time. Uber (the developer of Michelangelo), Airbnb (developer of zipline), and Google have all taken advantage of superior tooling and operations skills to build market-leading AI products.

Nearly every ML/AI library touts full end-to-end capabilities, from enterprise-ready stacks (such as H20.ai, MLFlow, and Kubeflow) to the highly specialized and engineer-friendly (such as Seldon.io) and everything in-between (like Dask). Enterprise level-frameworks often provide deep and well-supported integration with many common production systems; smaller companies might find this integration unnecessary or overly cumbersome. Regardless, it’s a safe bet that getting these off-the-shelf tools to work with your AI product in the exact ways you need them to will be costly (if not financially, then at least in time and human labor). That said—from a scale, security and features perspective—such capabilities may be required in many mature AI product environments.

On the other hand, building and scaling a software tool stack from scratch requires a significant sustained investment in both developer time and technology. Facebook, Uber, AirBnB, Google, Netflix, and other behemoths have all spent millions of dollars to build their ML development platforms; they also employ dozens to hundreds of employees, each tasked with building and scaling their internal capabilities. The upside here is that such end-to-end development to deployment frameworks and tools eventually become a competitive advantage, in and of themselves. However, it’s worth noting that in such environments, employing a single AI PM is not feasible. Instead, a cadre of PMs focused on different components of the AI product value chain are needed.

Where do we go from here?

Building great AI products is a significant, cross-disciplinary, and time-consuming undertaking, even for the most mature and well-resourced companies. However, what ML and AI can accomplish at scale can be well worth the investment.  Although a return on investment is never guaranteed, our goal is to provide AI PMs with the tools and techniques needed to build highly engaging and impactful AI products in a wide variety of contexts.

In this article, we focused on the importance of collaboration between product and engineering teams, to ensure that your product not only functions as intended, but is also robust to both the degradation of its effectiveness and the uncertainties of its operating environment. In the world of machine learning and artificial intelligence, a product release is just the beginning. Product managers have a unique place in the development ecosystem of ML/AI products, because they cannot simply guide the product to release and then turn it over to IT, SRE, or other post-production teams. AI product managers have a responsibility to oversee not only the design and build of the system’s capabilities, but also to coordinate the team during incidents, until the development team has completed enough knowledge transfer for independent post-production operation.

The evolution of AI-enabled product experiences is accelerating at breakneck speed. In parallel, the emerging role of AI product management continues to evolve at a similar pace, to ensure that the tools and products delivered to the market provide true utility and value to both customers and businesses. Our goal in this four-part series on AI product management is to increase community awareness and empower individuals and teams to improve their skill sets in order to effectively steer AI product development toward successful outcomes. The best ML/AI products that exist today were brought to market by teams of PhD ML/AI scientists and developers who worked in tandem with resourceful and skilled product teams.  All were essential to their success.

As the field of AI continues to mature, so will the exciting field of AI product management. We can’t wait to see what you build!

Sources:

• MLOps: Continuous delivery and automation pipelines in machine learning (Google)
• MLOps: What you need to know (Forbes)
• SLA vs. SLO vs. SLI: What’s the difference? (Atlassian)
• MLOps Tooling (Todd Morrill)
• Building Machine Learning Powered Applications (O’Reilly)
• Designing Data-Intensive Applications (O’Reilly)

Thanks

We would like to thank the many people who have  contributed their expertise to the early drafts of the articles in this series, including: Emmanuel Ameisen, Chris Albon, Chris Butler, Ashton Chevalier, Hilary Mason, Monica Rogati, Danielle Thorp, and Matthew Wise.

1. T-SQL in SQLite — CG/SQL is a code generation system for the popular SQLite library that allows developers to write stored procedures in a variant of Transact-SQL (T-SQL) and compile them into C code that uses SQLite’s C API to do the coded operations. CG/SQL enables engineers to create highly complex stored procedures with very large queries, without the manual code checking that existing methods require. (Open Source from Facebook)
2. Four Myths of Healthy Tech — (1) Social media is addictive, and we are powerless to resist it. The concept of addiction does not encompass the full range of pleasures, risks, and uses that people create with technology. (2) Technology companies can fix the problems they create with better technology. Some technology cannot be fixed by more design, and some technology should not be built at all. (3) Growth and engagement metrics are the best drivers of decision making at tech companies. Many of the most important parts of digital well-being cannot be captured by quantitative metrics. (4) Our health and well-being depend on spending less time with screens and social media platforms. Health and well-being cannot be reduced to the single variable of screen time. There’s detail to the alternatives presented to the myths, and it forms an interesting framework for thinking about “harmful social media”.
3. Telemelt — a web-based multi-emulator (RetroArch/libretro) designed to recreate the experience of playing console games with a single controller in a room full of friends.
4. Glitch Chasm — After the 212-story skyscraper in Melbourne, there’s a chasm to an airfield in the centre of the earth. Presumably some bad data for the airfield’s elevation.

Here's the 2 topics for this month's Virtual Meeting that you can attend on Thursday October 8th at 7PM by visiting: https://lufthans.bigbluemeeting.com/b/plu-yuk-7xx

der.hans: SSH Tunnels and More

Description:
Abstract: SSH is the go to tool for sysadmins and developers for interactive connections to remote machines. It creates secure, encrypted connections between computers, even across hostile networks. Secure unless you accept keys without verification (DON'T DO THAT!!!).

SSH can also create tunnels for encapsulating other connections, including other protocols and data. Sysadmins can bridge protocols across networks for ease of access such as a one-off data sync. Devs can present the dev database on their desktop to ease use of graphical development tools.

After attending this session, audience members will be able to create a local tunnel from client to server, a remote tunnel from server to client, and do simple analysis of local vs remote evaluation of a command. Attendees will be able to use tunnels for SSH or sample other protocols (MySQL and HTTP), and tunneling via a third party system. They will also be familiar with dynamic SOCKS proxies and using SSH to tunnel graphical applications. Finally, attendees will also learn SSH configuration and command line tips for convenience of use, including using forced command to restrict an SSH key to one purpose.

About der.hans:
der.hans is a technology and entrepreneurial veteran.

He is chairman of the Phoenix Linux User Group (PLUG), Promotions and Outreach chair for SeaGL, BoF organizer for the Southern California Linux Expo (SCaLE) and founder of the Free Software Stammtisch. He presents regularly at large community-led conferences (SCaLE, SeaGL, LFNW, Tübix, OLF, TXLF) and many local groups.

Currently a Customer Data Engineer at Object Rocket. Public statements are not representative of $dayjob.

Mastodon - https://floss.social/@FLOX_advocate

Plume - https://fediverse.blog/~/LuftHans

Sri- Circular Datacenter - Natural partners for FLOSS and Open Hardware

The Circular Datacenter is the idea that we can reuse datacenter
components and recertify them to new markets. This talk describes the
supply chain that the circular datacenter creates, and how we can leverage
that to build sustainable computational power that naturally allies itself
to free and open source software and open hardware.

The release of GPT-3 has reinvigorated a discussion of creativity and artificial intelligence. That’s a good discussion to have, primarily because it forces us to think carefully about what we mean when we use words like “creativity” and “art.” As I’ve argued in the past, each time we have this discussion, we end up raising the bar. Each time an AI system does something that looks “intelligent” or creative, we end up deciding that’s not what intelligence really is. And that’s a good thing. AI is likely to teach us more about what intelligence and creativity are not than about what they are.

I’m not terribly interested in whether AI can imitate human creativity. “Can an AI create a ‘new’ poem that reads as if it were written by Keats, or a new piano sonata that sounds like Beethoven” isn’t a question that’s worth asking. Of course it can—if not now, it will be able to in the near future. We really don’t need a new Beethoven sonata; the 32 he wrote are enough. Nor do we need a new Keats ode, limited though his output was. Or a new Rembrandt. Imitation is ultimately a party trick: clever and amusing, but not really important. Sure, if you want texts for greeting cards or elevator music (or maybe even commercial pop), algorithms may do the trick.

What’s really important is the transition between different forms of creativity. How do you get something that’s qualitatively new, and not just imitation? Creativity isn’t about the artifacts as much as it’s about the transitions. How do you get from Bach to Haydn? How do you get from Haydn to Beethoven? And, even within the career of a single artist: how do you get from the beginning to the end? How do you get from Beethoven’s first piano sonata, which sounds like Haydn, to the last, which at some points anticipates jazz? Artists aren’t stagnant. But I have no idea how even to ask whether an AI system can “mature” or “grow” in its output.

Great artists (and yes, I’m presuming a lot with the word “great”) frequently work by defining themselves against what came before. This is particularly clear with artists of the Romantic period in Germany, England, and France. The term Romanticism didn’t come around until some years later, but they left behind several manifestos describing what they were trying to do, and how it was different from what came before. That is still how artists work: Nnedi Okorafor’s Africanfuturism is important as a way of defining and directing her own work. The importance of these defining statements isn’t so much about “rightness” (in the sense of “this is what art is or should be”) but in setting a direction for their project. Can a machine do that? Can it decide how its work will be different from what came before? It’s not clear to me that it can’t, but that’s a significant step beyond any machine learning projects we currently have.

Although artists work by projecting their work into the future, by defining something “new,” their work is also derived from what came before—possibly as misinterpretation, but almost always as revision. Listen to the Beatles, and you hear something that was really built on the backbone of blues, filtered through some British pop-cultural trends. At the end of the “His Dark Materials” trilogy, Phillip Pullman thanks all the authors he stole from. Or, as T. S. Eliot said, “Immature poets imitate; mature poets steal; bad poets deface what they take, and good poets make it into something better, or at least something different.” Artists break from the past by reinterpreting that past.

For AI-generated art–where does that sense of “different” come from? Can artificial intelligence learn to steal and reinterpret? Where does that engagement with history, current events and even selfhood come from? Without that, there’s no basis for reinterpretation aside from random perturbation. It’s not clear that a sense of history couldn’t come from a big model trained on a gigantic corpus (although the best we can do now is build models that have no idea what they are saying). What kind of model would take that corpus and make something that was different, something that hadn’t been seen before? Would we care if Hamlet wasn’t written by Shakespeare in a specific historical context, but in 2025 by a computer trained on an archive of Elizabethan politics, drama, and history? (Never mind that an archive of Elizabethan drama would be very skimpy; most plays from that period were never published.) Would anyone care about the opera Nixon in China if it didn’t reflect a composer’s, and a librettist’s, thinking about historical events?

There’s another way in which I find computed-generated artworks unsatisfactory, particularly in music. AI-generated music is often interesting over the short term, but fails at larger-scale structure. Much of music history, from four-bar blues to Beethoven’s massive experiments in sonata form, is about building structures that can be interesting over the long term, whether “long term” means a few minutes of a blues song to several hours of opera. That was Beethoven’s project; more recently, it was the project of rock groups like Pink Floyd. It seems conceivable that a model could learn to generate such longer-form structures, but I’ve yet to see one that does it satisfactorily.

Is this where collaboration between humans and machines comes in, and if so, what does that say about creativity? A machine could conceivably do the pattern-matching and combinatorics, assembling a creative work out of news clippings and stylistic mimicry. But a human still needs to supply the sense of history that makes the work something we care about. A human still needs to provide the structure that makes art works more than brief curiosities. Is it possible for a human could tweak a model like GPT-3 to give it that sense of direction and context? What kinds of user interfaces would facilitate this kind of interaction?

I can’t answer those questions, but that sounds like a much more interesting form of digital collaboration than having an algorithm write hundreds of dull poems or songs and “curating” a few that aren’t boring. That’s just a recipe for greeting-card sentiment and elevator music. I mean no disrespect to Hallmark—mass-produced poetry for greeting cards serves a purpose—but when we think about what kinds of creativity we want, and how that creativity will be mediated by AI, we should demand more.

1. Algorithms Can Collude — To analyze the possible consequences, we study experimentally the behavior of algorithms powered by Artificial Intelligence (Q-learning) in a workhorse oligopoly model of repeated price competition. We find that the algorithms consistently learn to charge supracompetitive prices, without communicating with one another. The high prices are sustained by collusive strategies with a finite phase of punishment followed by a gradual return to cooperation. This finding is robust to asymmetries in cost or demand, changes in the number of players, and various forms of uncertainty. (via Marginal Revolution)
2. Hedgemony — Regular readers know I am fond of instructive games. RAND researchers developed Hedgemony, a wargame designed to teach U.S. defense professionals how different strategies could affect key planning factors in the trade space at the intersection of force development, force management, force posture, and force employment. The game presents players, representing the United States and its key strategic partners and competitors, with a global situation, competing national incentives, constraints, and objectives; a set of military forces with defined capacities and capabilities; and a pool of periodically renewable resources. The players are asked to outline their strategies and are then challenged to make difficult choices by managing the allocation of resources and forces in alignment with their strategies to accomplish their objectives within resource and time constraints.
3. Git Exercises — A repo that is its own set of git exercises.
4. Android in a Box — Run Android applications on any GNU/Linux operating system.

1. Single Device Behaves Like a Neuron — On its own, using a simple DC voltage as the input, the device outputs not just simple spikes, as some other devices can manage, but the whole array of neural activity—bursts of spikes, self-sustained oscillations, and other stuff that goes on in your brain. (Paper)
2. USB-C Is a Total Mess — Different power standards, differing video standards, and no way for a person to look at a cable or a connector and know what it can do. “The great thing about standards is that there’s so many to choose from.”
3. Unfck The Internet — Mozilla’s new campaign makes sense to me, but I can’t say it makes sense to launch during a pandemic election …
4. How Civil Society Can Combat Misinformation and Hate Speech Without Making It Worse — Good suggestions, backed by research. The six strategies for countering misinformation and hate speech: connected communities; the Truth Sandwich; pre-bunking; distributed debunking; localize the context; humor over rumor.

This month, the big surprise is that there’s no significant technology news about COVID. And there is more news than ever about legislation and regulation. I suspect that the legal system will be a big driver for technology over the next year. Another trend that doesn’t quite count as technology news but that definitely bears watching is that college enrollment in the US is down. Grad schools are up, 4 year colleges are down slightly; the big hit is in 2 year colleges. COVID is probably the biggest contributing factor, but regardless of the cause, this is an inauspicious trend.

AI and Data

• Beyond Text: Some experiments have trained GPT-3 on both text and images, enabling it to generate images from captions.
  
• The importance of time series: I’m less impressed with using time series to estimate the cost of an auto repair than with the possible applications of time series. Time series has had less attention than NLP, but it’s equally important.
  
• Can BERT learn common sense? Possibly. 50% correct answers to “common sense” problems doesn’t sound impressive, but I wonder how well humans would score.
  
• Training AI not to forget using generative replay techniques: One problem with neural networks is that they frequently “forget” important features in the data. Replay is an idea that is based on how humans remember: we can strengthen memories by recalling significant events.
  
• Microsoft Deep Speed uses new parallelism techniques to train very large models (billions of parameters) with less CPU and GPU power.
  
• Microsoft gets an exclusive license to OpenAI’s GPT-3: The public-facing API will remain, but access to the code is limited to Microsoft. As MIT Tech Review points out, this is a significant departure from OpenAI’s vision of freedom from both government and for-profit money.
  
• Biological signatures (e.g., heartbeat) can be used to detect deep fakes. It’s not news that your pulse can be detected in video. But the process of creating a fake distorts these signatures in ways that are detectable.
  
• Diffbot is building a knowledge graph of the entire web. It’s not a language model (though they may add a natural language query feature); it’s a database of facts in subject-verb-object form. Diffbot represents an older approach to machine learning that, in practice, might work better than language models like GPT-3.
  
• Visual reasoning with machine learning enables AI systems to reason about what people are doing and how they are doing it. This is an important step towards building robots that can function properly as assistants to humans.

Law

• Trump’s attempt to block WeChat has been blocked by the courts; now TikTok is in the same state. (The decision allows users to download TikTok; other restrictions that come into effect in November are still in place.) These blocks are only temporary, so presumably the future of these companies in the US is in the hands of the legal system.
  
• I have seen reports that the EU is interested in the TikTok deal for some unsurprising reasons. The EU is also concerned about social media companies headquartered in another superpower amassing their citizens’ data. Unintended consequences?
  
• Content moderation legislation: Brookings reports that legislation is increasingly focused on protecting aggrieved citizens than protecting innovation, and that legislation about content moderation is under consideration in Europe, Brazil, the US, and many other countries.
  
• There has been a lot of regulation surrounding privacy, but privacy regulation doesn’t address the problems raised by biometrics (including face recognition). A key issue is what happens when data is used outside of its original context: for example, when drivers license photos are used by agencies like ICE. AINow has published a report on the need for biometric regulation.

Virtual and Augmented Reality

• VR/AR glasses again: Facebook projects releasing the “next step on the road to augmented reality glasses” next year. It apparently won’t have a projector. One suspects it will have a camera, privacy concerns notwithstanding.
  
• The virtual trading floor: Investment companies are starting to use VR so traders can experience the trading floor while working from home. (Is this really a good idea?)
  
• Augmented reality for virtual geology probably isn’t a killer app, but it’s a useful tool for a science that can often involve a lot of travel. The authors say that AR leads to a better learning experience than VR because the students can actually see the teacher, rather than an avatar. Developing for holographic user interfaces may be the next step after Virtual Reality.

Cloud

• “Serverless 2.0” from Lightbend: Cloudstate implements function-as-a-service with stateful functions. It remains to be seen how well this will catch on, but until now, FaaS has required functions to be stateless.
  
• Microsoft’s Azure Arc lets you integrate on-premises servers or Kubernetes clusters with an Azure cloud. There’s been a lot of talk about hybrid cloud, but this strikes me as the most seamless integration possible.
  
• Google adds Confidential VMs and Confidential Kubernetes Engines to its cloud. These encrypt data as it’s in use (in memory), as distinct from encrypting data and storage (on disk) or in flight (on a network).
  
• Microsoft is experimenting with holographic storage for the cloud: extremely high volume, long-term storage.
  
• WebAssembly and Cloud Native: Could Wasm become the language for extending service meshes, allowing developers to write in any language they choose?  Google has been pushing this idea for Envoy and Istio.

Quantum Computing

• IBM’s Quantum Roadmap projects a 1000-Qubit machine by 2023. These are “physical” qubits, and may only be the equivalent of 50 or so logical qubits.  (The difference is due to error correction.) Google has published a similar roadmap, with practical quantum computers (1 million physical qubits) by 2029.

Security

• The Grugq’s definition of Cybercraft as a parallel to (and tool of) statecraft needs to inform thinking about security and operations. This is a consequence of a return to geopolitical competition that isn’t limited to the “great powers.”

Programming

• NVIDIA is acquiring ARM for $40B. This acquisition establishes NVIDIA as a powerhouse in general purpose computing, and as serious competition for Intel and AMD in their core businesses. Their intent is clearly to be a one-stop provider for AI and other applications that require high performance arithmetic.
  
• “Everyone will be a software engineer, and barely any will know how to code.”  A quirky but prescient article arguing that the key skill is understanding how to solve problems with software, not produce it. Certainly if no code and low code products democratize software creation, we’ll be left with the problem of understanding what the software should do.
  
• A video transcoder in the browser using Wasm is the most compelling demo of Wasm that I’ve seen yet. Wasm isn’t “coming”; it’s clearly already here. Developers just aren’t using it yet.

Cryptocurrency

• Many countries (the US is notably absent) are building pandemic-relief programs around central bank digital currency (cryptocurrency). This is a stepping stone to broader use of digital currency.

1. When Coffeemakers Demand Ransom — So he then examined the mechanism the coffee maker used to receive firmware updates. It turned out they were received from the phone with—you guessed it—no encryption, no authentication, and no code signing. Nothing remarkable here other than it’s 2020 and companies still put crappy software into their hardware.
2. Returns to Scale vs Experience — Big machines are sometimes more efficient. But they cost more, so fewer can be produced with a finite budget. Small machines are cheaper and may benefit from improvement over time, driven by experience in building more units. When does this experience lead to greater overall efficiency? We derive an approximation which, given a learning rate, tells how much smaller a machine must be to overcome an initial efficiency disadvantage.
3. Ten Years of Studying Social Bots — The first work that specifically addressed the detection of automated accounts in online social networks dates back to January 2010. A good history, courtesy the ACM.
4. CRDTs are the Future — A love note to CRDTs from someone who worked on Wave.

1. Adapton — A program P is incremental if repeating P with a changed input is faster than from-scratch computation. Adapton offers programming language abstractions for incremental computation.
2. Migration Lessons Learned — Keep your migration scripts away from your production code; Keep it low-tech, don’t deserialize; Write tests to exercise each migration script individually; Consider running long migrations online; Consider versioning your documents.
3. Microsoft Exclusive License to GPT-3 — If you’re selling compute, the logical complement is a clever system that sucks compute. I assume that’s why Oracle now have a slice of Tik-Tok. Capitalism is weird.
4. PNGlitch — However, we do not look at image formats from a general point of view, but rather think of ways to glitch them. When we look at PNG from the point of view of glitch, what kind of peculiarity does it have?

1. CS349 – Contemporary Issues in Computer Science — This class examines ethical frameworks, modern ethical concerns related to computer science and technology, and clear oral and written communication. Topics we will explore include policy vacuums created by new technology, copyright and patent, software bugs and liability, freedom of speech, privacy, security, employment and job markets, warfare and state-building, wealth discrepancy and consumerism, environmental impact, and changing cultural norms and social contracts. Wonderful to see this content being tackled in universities.
2. The Hardware Lottery — What follows is part position paper and part ahistorical review. This essay introduces the term hardware lottery to describe when a research idea wins because it is compatible with available software and hardware and not because the idea is superior to alternative research directions. We argue that choices about software and hardware have often played a decisive role in deciding the winners and losers in early computer science history.
3. Hope — a Bitsy game about learning to rely on others and fighting against hopelessness, together. I look for software that makes people and societies stronger, rather than weaker.
4. Your Values are the Rules You Break — If you are writing down “rules” and insisting that developers abide by them, it’s probably because your developers are continuously doing things you wish they wouldn’t. Usually, this isn’t because your developers don’t understand “the rules” and/or don’t like you—it’s because they know what the organization values, and those values are in conflict with your “rules,” and they’re trying to deliver that value.

1. A Concurrency Cost Hierarchy — a higher level taxonomy that I use to think about concurrent performance. We’ll group the performance of concurrent operations into six broad levels running from fast to slow, with each level differing from its neighbors by roughly an order of magnitude in performance. They are: Contended Atomics, System Calls, Implied Context Switch, Catastrophe, Uncontended Atomics, Vanilla Instructions.
2. Open Source Quadruped Robot — Now with a robotic arm.
3. AI Ethics Groups — without more geographic representation, they’ll produce a global vision for AI ethics that reflects the perspectives of people in only a few regions of the world, particularly North America and northwestern Europe. […] This lack of regional diversity reflects the current concentration of AI research (pdf): 86% of papers published at AI conferences in 2018 were attributed to authors in East Asia, North America, or Europe. And fewer than 10% of references listed in AI papers published in these regions are to papers from another region. Patents are also highly concentrated: 51% of AI patents published in 2018 were attributed to North America.
4. Threat Models for Differential Privacy — Looks at risks around central, local, and hybrid models of differential privacy. Good insight and useful conclusions, e.g. As a result, the local model is only useful for queries with a very strong “signal.” Apple’s system, for example, uses the local model to estimate the popularity of emojis, but the results are only useful for the most popular emojis (i.e. where the “signal” is strongest). The local model is typically not used for more complex queries, like those used in the U.S. Census [3] or applications like machine learning.

AI Benefits and Stakeholders

AI is a field where value, in the form of outcomes and their resulting benefits, is created by machines exhibiting the ability to learn and “understand,” and to use the knowledge learned to carry out tasks or achieve goals. AI-generated benefits can be realized by defining and achieving appropriate goals. These goals depend on who the stakeholder is; in other words, the person or company receiving the benefits.

There are three potential stakeholders for AI applications, with a single application often involving all three. They are business stakeholders, customers, and users. Each type of stakeholder has different and unique goals; each group is most interested in having their specific objectives met, or problems solved. My book, AI for People and Business, introduces a framework that highlights the fact that both people and businesses can benefit from AI in unique and different ways.

A typical social media platform needs to satisfy all three stakeholders. In the case of Twitter, the business stakeholder’s top goals are likely centered around profits and revenue growth. Customer stakeholders are the people and companies that advertise on the platform, and are most concerned with ROI on their ad spend. User stakeholders are interested in benefiting from the platform’s functionality: staying up-to-date, quickly finding new people and topics to follow, and engaging with family and friends.

Goals should be defined specifically and at a granular level for each stakeholder and relevant use case. Twitter has no doubt went through this exercise long ago; but if we imagine Twitter taking its first steps towards AI, some specific and granular goals could be to build a recommendation engine that helps users find the most relevant people to follow (a goal for users), while also building an AI-powered advertising targeting engine that best matches ads with those most likely to be interested in the product or service being advertised (for customers). This in turn would increase the platform’s value for users and thus increase engagement, which would result in more eyes to see and interact with ads, which would mean better ROI on ad spend for customers, which would then achieve the goal of increased revenue and customer retention (for business stakeholders). The key is to start with small and easily identifiable AI projects that will trickle value upwards towards a company’s highest priority goals.

AI Goals as a Function of Maturity

For companies early in their AI journey, setting appropriate goals helps create a foundation from which to build AI maturity. It also helps companies learn how to translate existing AI capabilities into solving specific real-world problems and use cases. In my book, I introduce the Technical Maturity Model:

I define technical maturity as a combination of three factors at a given point of time. These factors are:

• Experience: More experience usually results in increased muscle memory, faster progress, and greater efficiency. Teams with more experience with techniques such as natural language processing and computer vision are more likely to be successful building new applications using the same techniques. They’re not new to the field; they’ve solved problems, and have discovered what does and doesn’t work.
• Technical sophistication: Sophistication measures a team’s ability to use advanced tools and techniques (e.g., PyTorch, TensorFlow, reinforcement learning, self-supervised learning). When new tools appear, they can decide quickly whether they’re worth while, and get up to speed. They’re on top of the research, and are capable of evaluating and experimenting with new ideas.
• Technical competence: Competence measures a team’s ability to successfully deliver on initiatives and projects.  They have previously built similar, successful AI applications, and are thus highly confident and relatively accurate in estimating the time, effort, and cost required to deliver again. Technical competence results in reduced risk and uncertainty.

There’s a lot of overlap between these factors.  Defining them precisely isn’t as important as the fact that you need all three. Higher levels of experience, technical sophistication, and technical competence increase technical maturity. Increased AI technical maturity boosts certainty and confidence, which in turn, results in better and more efficient AI-powered outcomes and success.

Technical maturity is a major factor behind why some companies are very successful with AI, while other companies struggle to get started and/or achieve success.

The Challenge with Defining AI Goals

Turning an AI idea into actual benefits is difficult and requires the “right” goals, leadership, expertise, and approach. It also requires buy-in and alignment at the C-level.

Identifying, prioritizing, and goal-setting for AI opportunities is a multi-functional team effort that should include business folks, domain experts, and AI practitioners and researchers. This helps ensure alignment with company goals, while also including necessary business and domain expertise. AI initiatives may also require significant considerations for governance, compliance, ethics, cost, and risk.

Further, while the technical details of AI are complex, the outputs of AI techniques are relatively simple. In most cases, AI solutions are built to map a set of inputs to one or more outputs, where the outputs fall into a small group of possibilities. Outputs from trained AI models include numbers (continuous or discrete), categories or classes (e.g., spam or not-spam), probabilities, groups/segments, or a sequence (e.g., characters, words, or sentences).

Therefore, AI techniques don’t just solve real-world problems out of the box. They don’t automatically generate revenue and growth, maximize ROI, or keep users engaged and loyal. Likewise, AI doesn’t inherently optimize supply chains, detect diseases, drive cars, augment human intelligence, or tailor promotions to different market segments.

Setting a company-wide goal of reducing customer churn by 25% is great, but, unfortunately, is far too broad for most AI applications. That’s why customer churn reduction is not a natural output of AI techniques. The mismatch between goals like reducing customer churn and actual AI outputs must be properly handled and mapped.

Why and How to Set Good AI Goals

AI goals should be appropriate for a given company’s technical maturity, and should be chosen to maximize the likelihood of success, prove value, and build a foundation from which to create increasingly sophisticated AI solutions that achieve higher-level business goals. A crawl, walk, run approach is a good analogy for this.

Goals should be well-formed, meaning they are stakeholder-specific, map actual AI outputs to applications and use cases that achieve business goals, and are appropriately sized. For companies early in their AI maturity, appropriately-sized goals mean that they should be small and specific enough to experiment with, and prove potential value from, relatively quickly (think lean methodologies and incremental). As AI maturity increases, a non-incremental, holistic, and organization-wide AI vision and strategy should be created to achieve hierarchically-aligned AI goals of varying granularity—goals that drive all AI initiatives and development. This should be accompanied by a transition from incremental thinking to big vision, “applied AI transformation” thinking.

Let’s consider the overall goal of reducing customer churn. In an early stage of AI maturity, we can build AI solutions that reduce search friction (e.g., Netflix and Amazon recommendation engines), increase stickiness through personalized promotions and content that is more relevant and engaging, create a predictive model to identify customers most likely to churn and take appropriate preventative actions, or automate and optimize results in areas that are outside of a person’s primary area of expertise (e.g., automated retirement portfolio rebalancing and maximized ROI). When transitioning to developing a bigger AI vision and strategy, we may create a prioritized product roadmap consisting of a suite of recommendation engines and an AI-based personalized loyalty program, for example.

At the individual goal level, and for each well-formed goal, the same multi-functional team mentioned earlier must work collaboratively to determine what AI opportunities are available, select and prioritize the ones to pursue, and determine the technical feasibility of each.

There are frameworks like SMART to help characterize well-formed goals, but since AI is a field that I characterize as scientific innovation (like R&D), characteristics like being achievable and time-bound may not be the best goals. Results are typically achieved through a scientific process of discovery, exploration, and experimentation, and these processes are not always predictable.

Given the scientific nature of AI, goals are better expressed as well-posed questions and hypotheses around a specific and intended benefit or outcome for a certain stakeholder. With well-formed goals, data scientists and machine learning engineers can then apply the scientific method to test different approaches in order to determine the validity of the hypothesis, and assess whether a given approach is feasible and can achieve the goal.

For example, by introducing the “Frequently bought together” recommendations (and other recommendations), Amazon was able to increase average customer shopping cart size and order amount (i.e., up-sell and cross-sell), which in turn increases average revenue per customer, which in turn increases Amazon’s e-commerce generated revenue per quarter. McKinsey estimates that up to 35% of Amazon’s revenue and 75% of everything watched on Netflix comes from AI-powered recommendations.

But when defining an AI project, the goal or hypothesis in this case isn’t to increase top-line revenue for the company, but rather to posit that building an application that groups products by likelihood to be purchased together will increase average customer order size, which in turn will have an upward impact on top level goals like increasing average revenue per customer and top-line revenue.

Another example would be setting a goal around building a well-performing AI model that can predict demand (number of units likely to be purchased) for a specific product for a given day, time, and weather conditions. If accurate, this prediction can help a retailer ensure that they do not run out of stock, which means that there is no lost revenue because a product is out of stock. An added benefit is improved customer experience, which results in happier and more loyal customers who are able to buy the products they want whenever they want to buy it. This same approach can be applied to virtually any other application of AI.

Conclusion

AI and machine learning technologies have come a long way in terms of capabilities and accessibility, but off-the-shelf AI solutions aren’t yet available for specific industries or business domains, companies, sets of data, applications, and use cases. The key to success with AI is assembling a multi-functional team that defines appropriate goals, then letting these goals drive the AI initiatives and projects.

1. Accurately Lipsync Video to Any Speech — In our paper, A Lip Sync Expert Is All You Need for Speech to Lip Generation In the Wild, ACM Multimedia 2020, we aim to lip-sync unconstrained videos in the wild to any desired target speech. (Paper) Impressive.
2. Temporal — Open source “workflow-as-code” engine. I can’t decide if this is awful or brilliant.
3. The Rate of Obsolescence of Knowledge in Software Engineering — When I graduated, I was told the half-life for what I’d learned was 18 months. But not all knowledge is equivalent, as the author points out. The anatomy of an OS class is still relevant today. It’s interesting to look at the knowledge you struggle to acquire and ask yourself what its half-life will be.
4. Security Engineering (3ed) — Drafts of the third edition, to be released in December, are available online but may go away. (via Bruce Schneier)

We have a couple of presentations lined up this month for you to enjoy from the safety of your home.

Attend by going to: https://lufthans.bigbluemeeting.com/b/plu-yuk-7xx

Adrian Cochrane: What is the small web And Why Is It Important?

Description:
To support the creation of "webapps" web browsers have become increasingly complex, to the extent that even major corporations can no longer continue keeping up.

There's been a growing community concerned about this direction the web's been taking and wanting to do something about. This talk will show that this is possible and vital to do.

About Adrian:
Adrian started programming when he was ten years old, when he'd sleep with a Python book under his pillow. Since then he continued studying programming eventually graduating from Victoria University of Wellington with a BSc Computer Science, now running a contracting business with his father supporting the establishment of open standards.

He started developing his own browser to explore an increasing fascination with how to (re)discover valuable webpages, occasionally contributing code to related projects. And frequently studying all the code he can about how elementary OS works.


Kevin Tyers: BASH - Practical Tips

Description:
Kevin will be sharing 6 of his most beloved tricks for using Bash. These simple, yet practical tips should be immediately useful, or at least inspirational for you to develop your own set. This talk is for people of all skill levels.

About Kevin:
Kevin Tyers is a SANS Instructor, the head of cyber intelligence engineering for a Fortune 250 company, and the head of infrastructure for iCTF. Throughout his 15-year career, he has worked in the government, telecom, health care, and financial industries focusing on network engineering/security, incident response, and tooling. Kevin is the cofounder of the Information Security group DC480 in Phoenix Arizona. He has spoken at a variety of public and invite-only conferences such as BSidesLV, CactusCon, and SANS Hackfest. He has been a Linux user for as long as he can remember and is passionate about sharing the tips and tricks he has learned for using Linux.

1. Things I Learned to Become a Senior Software Engineer — Full of relatable growth moments, such as changing your code to make the test pass vs understanding why the test failed.
2. The Future is Software Engineers Who Can’t Code — “There are lot of definitions of what a developer is […] It’s not just people who write code.” […] Microsoft has even given these “civilian” programmers a persona: Mort. […] The fictional “Mort” is a skilled professional, anyone from a business analyst to a construction site cost estimator, who needs computers to perform specific functions without mastering the intricacies of full-blown programming. As Mel Conway called it, the profession is bifurcating into architects and plumbers. Architects make complex pieces of software, plumbers bolt those pieces together.
3. Pair Programming with AI — This makes sense to me: We don’t need complete, perfect solutions; we need partial solutions in situations where we don’t have all the information, and we need the ability to explore those solutions with an (artificially) intelligent partner.
4. Writing System Software: Code Comments — Absolutely the best thing on software engineering that a software engineer will read all month. This is GOLD.

In a conversation with Kevlin Henney, we started talking about the kinds of user interfaces that might work for AI-assisted programming. This is a significant problem: neither of us were aware of any significant work on user interfaces that support collaboration. However, as software developers, many of us have been practicing effective collaboration for years. It’s called pair programming, and it’s not at all like the models we’ve seen for interaction between an AI system and a human.

Most AI systems we’ve seen envision AI as an oracle: you give it the input, it pops out the answer. It’s a unidirectional flow from the source to the destination. This model has many problems; for example, one reason medical doctors have been slow to accept AI may be that it’s good at giving you the obvious solution (“that rash is poison ivy”), at which point the doctor says “I knew that…” Or it gives you a different solution, to which the doctor says “That’s wrong.” Doctors worry that AI will “derail clinicians’ conversations with patients,” hinting that oracles are unwelcome in the exam room (unless they’re human).

Shortly after IBM’s Watson beat the world Jeopardy champions, IBM invited me to see a presentation about it. For me, the most interesting part wasn’t the Jeopardy game Watson played against some IBM employees; it was when they showed the set of answers Watson considered before selecting its answer, weighted with their probabilities. That level was the real gold.  We don’t need an AI system to tell us something obvious, or something we can Google in a matter of seconds. We need AI when the obvious answer isn’t the right one, and one of the possible but rejected answers is.

What we really need is the ability to have a dialog with the machine. We still don’t have the user interface for that. We don’t need complete, perfect solutions; we need partial solutions in situations where we don’t have all the information, and we need the ability to explore those solutions with an (artificially) intelligent partner. What is the logic behind the second, third, fourth, and fifth solutions? If we know the most likely solution is wrong, what’s next? Life is not like a game of Chess or Go—or, for that matter, Jeopardy.

What would this look like? One of the most important contributions of Extreme Programming and other Agile approaches was that they weren’t unidirectional. These methodologies stressed iteration: building something useful, demo-ing it to the customer, taking feedback, and then improving. Compared to Waterfall, Agile gave up on the master plan and specification that governed the project’s shape from start to finish, in favor of many mid-course corrections.

That cyclic process, which is about collaboration between software developers and customers, may be exactly what we need to get beyond the “AI as Oracle” interaction. We don’t need a prescriptive AI writing code; we need a round trip, in which the AI makes suggestions, the programmer refines those suggestions, and together, they work towards a solution.

That solution is probably embedded in an IDE. Programmers might start with a rough description of what they want to do, in an imprecise, ambiguous language like English. The AI could respond with a sketch of what the solution might look like, possibly in pseudo-code. The programmer could then continue by filling in the actual code, possibly with extensive code completion (and yes, based on a model trained on all the code in GitHub or whatever). At this point, the IDE could translate the programmer’s code back into pseudo-code, using a tool like  Pseudogen (a promising new tool, though still experimental). Any writer, whether of prose or of code, knows that having someone tell you what they think you meant does wonders for revealing your own lapses in understanding.

MISIM is another research project that envisions a collaborative role for AI.  It watches the code that a developer is writing, extracting its meaning and comparing it with similar code.  It then makes suggestions about rewriting code that looks buggy or inefficient, based on the structure of similar programs. Although its creators suggest that MISIM could eventually lead to machines that program themselves, that’s not what interests me; I’m more interested in the idea that MISIM is helping a human to write better code. AI is still not very good at detecting and fixing bugs; but it is very good at asking a programmer to think carefully when it looks like something is wrong.

Is this pair programming with a machine? Maybe.  It is definitely enlisting the machine as a collaborator, rather than as a surrogate. The goal isn’t to replace the programmers, but to make programmers better, to work in ways that are faster and more effective.

Will it work? I don’t know; we haven’t built anything like it yet. It’s time to try.

1. Inside the Digital Pregnancy Test — … is a paper pregnancy test and watch-battery-powered microcontroller connected to three LEDs, a photo-cell, and an LCD display. That (8-bit) microcontroller runs at 4MHz, almost as fast as an IBM PC did.
2. The Incredible Proof Machine — Fun game (modelled on The Incredible Machine from the 90s) that teaches logic.
3. Make Interfaces Hard to Misuse — Don’t push the responsibility of maintaining invariants required by your class on to its callers. Excellent advice.
4. ArangoDB — a scalable open-source multi-model database natively supporting graph, document and search. All supported data models & access patterns can be comnbined in queries allowing for maximal flexibility.

1. VSCode Debug Visualizer — A VS Code extension for visualizing data structures while debugging. Like the VS Code’s watch view, but with rich visualizations of the watched value. The screencast is wow.
2. Userland — an integrated dataflow environment for end-users. It allows users to interact with modules that implement functionality for different domains from a single user interface and combine these modules in creative ways. The talk shows it in action. It’s a spreadsheet and cells can be like a spreadsheet, or can be like a Unix shell, or can be an audio synthesizer (!).
3. Minglr — Open source software (built on Jitsi) that facilitates the ad hoc mingling that might happen in the audience after a talk ends: see who’s there, pick people to talk to, talk to them. Interesting to see the floresence of social software thanks to lockdown.
4. Crepe — a library that allows you to write declarative logic programs in Rust, with a Datalog-like syntax. The Reachable example is sweet. From initial testing, the generated code is very fast. Variants of transitive closure for large graphs (~1000 nodes) run at comparable speed to compiled Souffle, and at a fraction of the compilation time.

Compared to the last few months, there are relatively few items about COVID. And almost no items about Blockchains, though the one item I’ve listed, about China’s Blockchain Services Network, may be the most important item here. I’m seeing a steady stream of articles about various forms of no-code/low-code programming. While many programmers scoff at the idea of programming-without-programming, spreadsheets are an early example of low-code programing. Excel is hardly insignificant. On the AI front, the most significant change is discussion (see the thread below) of a “Deep Learning Recession,” as companies under pressure from COVID look for results and can’t find them.

AI

• There is serious talk of a “Deep Learning recession” due, among other things, to a collapse in job postings. Short-term effect of COVID or long term trend?
  
• An excellent analysis of participation in machine learning: how it is used, and how it could be used to build fair systems and mitigate power imbalances.
  
• Fairness and Machine Learning is an important new (academic) book by Solon Barocas, Moritz Hardt, and Arvind Narayanan. It’s currently an incomplete draft, available (free) online.
  
• A draft document from NIST describes Four Principles of Explainable Artificial Intelligence. The ability to explain decisions made by AI systems is already important, and will become more so.
  
• SAIL-ON is a DARPA-funded research project to develop AI systems that can deal with novelty (such as the COVID pandemic), starting with unexpected situations (such as changes to the rules) in board games.
  
• Is NLP research pursuing the right goals? While GPT-3 is impressive, it doesn’t demonstrate anything like comprehension (ordering the relationships found in a story). (David Ferrucci, Elemental Cognition). Likewise, Gary Marcus argues that GPT-3 can put together sentences that sound good, but that it has no idea what it’s talking about.
  
• What happens when you combine a relational database with git? You get a Dolt, a database that enables collaboration on datasets.  You might get a solution to the problem of data versioning–and a big step towards CI/CD pipelines for AI applications.

COVID-19

• Cough recognition? AI to locate people who cough in space and time. A very questionable tool for pandemic fighting.
  
• Patient-led research on COVID-19 is an organization to help long-term COVID patients share their observations.  This is reminiscent of PatientsLikeMe, and related to other trends in re-envisioning healthcare.

Programming

• Turning a Google Sheet into an app without code is yet another example of the low-code trend.
  
• Chris Lattner (one of the co-creators of LLVM and of Swift) has an interesting AMA that, among other things, talks about integrating machine learning with compilers, and machine learning as its own programming paradigm.
  
• Contextual engineering is fundamentally simple: consider “why” before thinking about “how.”  But ignoring “why” is at the heart of so many engineering failures throughout history. And understanding “why” often requires “immersion in the local culture.”  This is starting to sound like an extended version of bounded context.
  
• Blazor is a new framework for the Web that allows developers to program with C#, compiling to Web Assembly (wasm).  It’s potentially a competitor for JavaScript and React/Angular/Vue, though it may have trouble spreading outside of the Microsoft community.

Cloud and Microservices

• K3s is a stripped-down Kubernetes designed (among other things) for IoT and Edge Computing.  I’ve thought for some time that Kubernetes needs simplification. Is this it?
  
• Microsoft announces Open Service Mesh for managing communications between microservices. OSM is based on the Service Mesh Interface, and is an alternative to Google’s Istio, which has a reputation for being difficult, and has become controversial.
  
• SMOKEstack is Redmonk’s “alternative stack” for multi-cloud environments (and perhaps doing an end-run around Amazon’s hegemony). SMOKE stands for Serviceful, Mashable, Open, K(C)composable, Event-driven.
  
• IBM’s RoboRXN is a “chemistry lab in a cloud” that’s designed for drug synthesis; you design an experiment, which executes in a robotic lab. The idea is similar to Max Hodak’s Transcriptic (now Strateos; Max is now founder and president of Elon Musk’s Neuralink).  But IBM adds some twists: you design a molecule with a graphical (low-code) interface, and the actual process is filled in using AI.

Quantum Computing

• Amazon’s Braket service: true quantum computing in the cloud, and now available to the public. IBM and Microsoft already have quantum computers available through their cloud offerings; Google will eventually follow. We’re still at the tire kicking stage, since none of these machines can do real work yet.
  
• NIST has announced a number of cryptographic algorithms that can’t currently be broken by quantum computers. This is a significant step towards quantum-proof encryption.

New Infrastructure

• TRUSTS is a data market funded by the European Commission.  MIT Tech Review has a good explanation. It’s a little surprising to see this coming out of the EU, but once people have the right to privacy, the right to sell data is not far behind. Individuals don’t participate in the market as individuals; the trust handles (and enforces) the transactions and pays dividends.
  
• One of the biggest problems with privacy and identity has been developing the infrastructure for managing public keys. Sidetree is a protocol for decentralized public key infrastructure based on a blockchain.
  
• China’s blockchain infrastructure (BSN) supplies the infrastructure for a global financial services network.  It is not a blockchain, but a network for building interoperable blockchain applications that is targeted at small to medium businesses, with the intent of making RMB a viable global currency.  The US Federal Reserve has also released plans for cryptocurrency: They are planning to launch a service called FedNow in 2-3 years. They are way behind the Chinese.

Operations

• A pre-release of a paper describes zero-downtime deployments at Facebook scale. There’s a good thread on Twitter discussing the paper.

Social Media

• An algorithm for controlling fairness and bias in search results rations exposure, preventing a popular link from gaining clicks relative to similar links, while still maximizing usefulness.
  
• Twitter has released their new API. After abusing their developers a decade ago, will this make any difference?
  
• In iOS 14, Apple will be requiring opt-in for tracking users’ web activity. Facebook is not happy about this; targeted advertising depends critically on user tracking. Google (which has been gradually implementing other limitations on advertising technology) has been quiet about it.