AI News, Most frugal explanations in Bayesian networks

JMIR Publications

Most of the daily news and recently published scientific papers on research, innovations, and applications in artificial intelligence (AI) refer to what is known as machine learning—algorithms using massive amounts of data and various methodologies to find patterns, support decisions, make predictions, or, for the deep learning part, self-identify important features in data.

In more than 60 years of exploration and progress, AI has become a large field of research and development involving multidisciplinary approaches to address many challenges, from theoretical frameworks, methods, and tools to real implementations, risk analysis, and impact measures.

As a consequence of the wide landscape, the field draws at large through philosophy, mathematics, information sciences, computer science, psychology, anthropology, social sciences, linguistics, and many others.

For some experts and visionary people such as Ray Kurzweil, deep machine learning will allow building of an artificial general intelligence that is able to develop itself autonomously and to have the capacity to understand or learn any intellectual task that a human being can, and even go far beyond the limits of human intelligence [5], but most experts would agree that there are some big missing pieces and it is still a long way off, despite recent potential important advances in quantic computing [6].

A recent white paper published by the European Commission and authored by the members of the High-Level Expert Group on AI provides, in a few pages, a good overview on what AI is, its main capabilities, applicable expectations, and disciplines involved [7].

Advanced signal processing is implemented in pacemakers or defibrillators to take decisions, in cochlear-implants with man-machine interfaces, in electrocardiograms to provide signal analysis and automated diagnosis, etc.

Machine learning and deep learning has led most recent major breakthroughs in AI, such as sound (speech and music) recognition and image (face, radiology, pathology, dermatology, etc) recognition, and in gaming.

This question is defined as determining the positive predictive value of a positive test, and it gives the probability of a positive test to be really signing the presence of the factor the test is testing.

At the other end, if the prior probability is around 20% (ie, a woman with several factors suggesting a potential pregnancy), the probability of a positive test to be a true positive is above 80%.

This has been shown well with decision support system in CPOE, with a very high rate of false-positive alerts, especially with patients receiving complex drug therapies [10,11].

In addition, medicoeconomic assessments are often used by health agencies according to various dimensions such as quality-adjusted life year and burden of the disease, by using indicators such as disability-adjusted life-years [12-14].

This is an important aspect, as regulatory agency support is an important asset in building trust for most care professionals to use medical tools and for companies to invest in robust products ready for the market.

In 5-10 years, when current young students will be starting their clinical activities, machine learning based on data science will have become embedded in many activities, devices, and software and its use, misuse, and overuse and consequences on patients and accountability will depend on how users will master it [23].

Designing long-term cohorts and building metadata framework and standard operating procedures for biobanking are important challenges, as they have to project usages that will be made years after the initial design.

This assessment has been adapted to reflect changes in radiological response, for example, in immunotherapies where the size of tumors can increase despite good therapeutic response [32].

Unfortunately, there is growing pressure to extend the use of RECIST and other similarly structured staging guidelines beyond clinical trials for all radiological staging to improve the capacity to use standard clinical care for therapeutic assessment.

As a consequence, this leads to a high time pressure on operational activities of radiology departments and an increasing number of inexperienced people using these types of staging.

With the progression of natural interfaces such as voice recognition and natural language processing and their increased daily use in a growing number devices, I would argue in favor of avoiding artificial structuring many data acquisition processes and keep the data in their most natural form, exploiting more natural interactions such as voice and text and developing strong natural language processing tools that can be applied to produce structured information in a postprocessing step.

The same applies with rule-based techniques or symbolic reasoning, which need to be able to express rules, that is, truth in a formalized way, but also in supervised machine learning approaches, which require having training sets that express truth, at least a probabilistic truth.

There are a lot of expectations in these approaches, especially when combing them [35,36], but all of them, except unsupervised deep machine learning, require some sources of truth, which leads to the fundamental question of finding the sources of truth in life sciences and the level of evidence supporting that truth.

However, a recent paper from Rizwan et al [38] describes a new type of lymphocyte, bearing characteristics of both B and T cells, which may play a role in driving autoimmunity in some diseases such as diabetes [38].

The latter is currently the object of numerous works, trying to understand intermediate representation of data in neural networks that can predict and explain their behavior.

For example, in Science in 2018, Hutson [46] reported a survey of 400 artificial intelligence papers presented at major conferences, with only 6% including code for the algorithms and 30% test data, thus considerably limiting reproducibility possibilities [46].

Increasing heterogeneous data sources and richness of data about each of us, associated with data linkage techniques, strongly increases the possibility of reidentification, including anonymized data [52-57].

There is no good technical solution that can harmonize the challenge of preserving privacy and answering the increasing need of data-driven science for accessing large genomic et phenotypic datasets, and there are many ongoing ethical and legal discussions [61-66].

An important step forward is to improve awareness and education of all stakeholders about privacy, technical limitations to protect it, and building regulatory barriers to avoid discrimination.

There are several important initiatives that contribute to this, such as the Global Alliance for Genomics and Health (GA4GH), an organization setting a policy and technical framework for respecting human rights to enable responsible genomic data sharing [74], or the European Union General Data Protection Regulation (GDPR) [75] that sets a completely novel privacy regulation for the European Union.

Current DPhil Projects

The knowledge of the output range for the network activations is essential to fully understand the resulting system, and many verification problems can be formulated as optimisation problems to compute network bounds.

While being interesting and important in its own right, a grounded understanding of deep learning is essential if modern machine intelligence is to be safe and interpretable.  Moreover, increased clarity on the inner workings of deep learning algorithms has potential to guide practitioners in advancing state of the art techniques and applications.

The two companies are hesitant to share data and proprietary information, so there is a need for a third party to design a system that allows all companies to operate in the same space, achieve their respective individual goals, and avoid collisions.

Further goals include modifying the proposal to be online, in which case it must be flexible under changes in environment and individual goals before completion and robust to uncertainty in individual transitions during path execution.

The challenge of designing agents that are capable of innovation has two facets.  The fi rst is to create spaces which correspond to ideas or candidates for the task, such as possible tools for a robot reaching task.

In the context of visual data however (images and videos), computer vision methods still lack meaningful tools for comparison relative to human visual perception.

For example, a child which sees a single picture of a zebra at the end an alphabet book, will often be able to recognise the animal in a zoo, despite inevitable differences in pose, appearance, scale and other quantities.

This research aims to leverage self-similarity present in natural images to solve multiple tasks in the supervised-learning framework and to develop representations of visual data which can be widely applied to multiple computer vision tasks.

The second project was titled “ADMM for Control of Mixed Traffic Flow with Human-driven and Autonomous Vehicles”, which defined a dynamical system for a multi-vehicle traffic problem and created a control strategy for the autonomous vehicle to reduce traffic congestion.

Humans, on the other hand, learn a great deal from associations between senses: for example, early in development, seeing a face and hearing a voice teaches us about other people’s presence and identities.

As such, it is natural to ask questions like ‘how can this fleet of autonomous cars get from point A to their respective destinations both safely and in the least amount of time?’ or ‘how can these industrial robots perform their respective functions, whilst collectively expending the least amount of battery power?’.

Whilst the verification of temporal properties of systems where the agents have mean-payoff rewards has been looked at be- fore, we believe this will be the first work to directly incorporate both temporal goals and numerical payoffs into the rewards of the agents.

Deep model architecture design in its current form is ambivalent to its resource requirements.  Consequently, the state-of-the-art models are too large and too costly to run on most everyday hardware.  This has very serious and negatove implications for the field.

First, the current situation monopolizes the technology as only very few very large corporations and governments have the ability to deploy and monetize these models.  Unless access is broadened the deep product development wil be dominated by a small number of large players - with the predictable monopoly effect of slower innovation, higher proces, and lower quality.  Second, the current state introduces serious data privacy concerns as data needs to be transmitted and centralized.  The centralized introduces a single point of attach, while the transmission exposes potentially highly priovate data to the public.  Both need to be avoided or mitigated in the design of a secure system.  Finally, the current modus operandi prohibits some of the most important applications from being developed.  Particularly embedded systems stand to gain the most from running models locally - this couls be anything from the deep learning powerered robotic arm in space to the smart insulin dispenser off the grid.

robotics) and multi-agent communication depend on “good” representations in order to solve the downstream tasks and usually such mapping is created manually by experts who incorporate domain knowledge (feature engineering).

More and more people turn to self-employment, while businesses seem to be exploring recent studies related to increasing productivity, by allowing their employees to have flexible working times, of­ten working from home.

even though nowadays older retired adults normally have many more years to live, they are often faced with age-related diseases, such as arthritis, Parkinson’s, dementia or geriatric depression, that might keep them at home as they become more severe.

As the field of machine vision has been at the forefront of neural network research, has standard testing data sets and well established baselines, this field is well suited for starting this line of research.

In order to develop better relationships with robots, in particular in a care or hospital environment, these robots should appear more natural in their movements as well as be able to perform useful tasks.

Much of the research in robotic control aims to develop solutions that, depending on the environment of operation, exploit the machine’s dynamics in order to achieve a highly agile behavior.

This, however, is limited by the use of traditional control techniques such as model predictive control (MPC) [1] and quadratic programming (QP) [2] which are often based on simplified rigid body dynamics and contact models.

A model-based optimization strategy employed over such simplified models often results in a constrained range of solutions that do not fully exploit the versatility of the robotic system, thereby limiting the agility of the robot in question.

Treating the control of robotic systems as an RL problem enables the use of model-free algorithms that attempt to learn a policy which maximizes the expected future (discounted) reward without infer­ring the effects of an executed action on the environment.

Authors of [3] [4] and [5] have successfully implemented these strategies for various robotic applications including control of robotic manipulators, helicopter aerobatics, and even quadrupedal locomotion.

However, despite the successful implementa­tion of these RL algorithms for the mentioned tasks, one of the main challenges faced in solving an RL problem is defining a reward function in order to learn an optimal policy resulting in a sensible robotic behavior.

For tasks such as quadrupedal navigation through rough terrain, computing a reward function is also significantly more difficult than for tasks such as posture recovery, which when solved using an RL algorithm results in a near-optimal policy.

For instance, if a vision component has a slight error on an object position, the manipulation component would attempt to pick it up at the wrong position and would have no clue how to correct it. Learning vision and control in an end-to-end manner gives us the opportunity to overcome this difficulty and has emerged as a new trend.

In my research I will focus on improving existing branch and bound methods that exploit the piecewise linear structure of neural networks with the aim of being able to apply them to larger networks.

Improvements can be made to all three parts of the branch and bound algorithm: the search strategy, which picks the next domain to branch on, the branching rule, which given a domain divides it into non-intersecting subdomains, and finally the bounding methods which estimate lower and upper bounds for each subdomain.

To cope with such challenges, geometric deep learning (GDL) [1] is a branch of emerging deep learning techniques that makes use of novel concepts and ideas brought about by graph signal processing (GSP) [2], a fast-growing eld by itself, to generalise classical deep learning approaches to data lying in non-Euclidean domains such as graphs and manifolds.

This typically involves image alignment or registration which can be described as the task of inferring correspondences and transformations that map images to the same coordinate system and therefore emphasise change.

While the previous methods have shown considerable success in the well-defined cases, the problem is far from being solved in the ambiguous examples presenting texture-less regions or repetitive patterns.

Whereas change induced by camera motion is generally considered extraneous and can be compensated through robust image alignment, the notion of relevance of change may vary according to applications, which makes the problem ill-defined.

This has motivated many people to employ these techniques to various robotics tasks including autonomous driving which incorporates many classical vision problems such as segmentation, classification, depth prediction, and uncertainty estimation.

Many of these recent techniques in machine vision leverage large convolutional neural networks (CNNs) that require graphics processing units (GPUs) to both train and run at inference time because of their large computational load.

As a starting point this research will look to develop novel methods for training Binary and Quantised Neural networks by using discrete programming relaxations to train binary neural networks.

If comparable results to modern CCNs could be replicated on low powered CPUs such as those found in mobile devices this would have a huge impact on the areas of self-driving cars, robotics, smart data acquisition and portable AI.

We want to build off of this work by incorporating ideas from the optimal dataset selection literature [7] and to incorporatemore prior knowledge into the procedure through better handling of certain hyperparameters with a-priori known effects e.g.

Looking further afield we want to address other problems in optimization such as neural network architecture selection [8] and address problems in numerical quadrature, such as evaluation of model likelihoods [1].

Probabilistic machine learning uses probability theory to represent and manipulate uncertainty and is based on the idea that learning can be thought of as inferring plausible models to explain observed data.

The former research focus will help open up a new range of problems to which reinforcement learning can be applied, while the latter will make training in reinforcement learning and meta-learning amenable to a wide range of probabilistic inference methods.

Machine learning has made remarkable progress in recent years by exploiting 'deep' models, which promise to learn complex representations of their input, aiming to discover the underlying structure of the problem directly from data.

Even in toy cases where a very simple invariance in the data exists, empirically deep models do not always infer it even in the limit of large amounts of data, showing failure to learn even simple structure.

In addition, models are often sensitive to extremely small pertubations to their input, which show that they often achive their performance by using features not semantically relevant to the task at hand.

Probabilistic modelling is one solution to enforcing model structure.  However, it is challenging - often structured models are too restrictive, and as a result are extremely difficult to fit.  This is especially true for data like images, where specifying a direct likelihood over pixels is often both restrictive and artificial.

We think exploring ways to both add more flexibility to structured probablistic models and to use newer techniques, like GANS, to learn more traditional graphical models will be a fruitful area of research.

Probabilistic programming simplifies the use of probabilistic modelling thanks to the ease of defining generative models, and saves the effort of deriving custom inference algorithms for the model of interest thanks to the general purpose Monte Carlo or black box variational inference algorithms which are available as part of some prominent probabilistic programming languages or systems, such as Anglican.

In order to make them both scalable and accessible we require a new approach to the current paradigm, that combines human domain knowledge and a hybrid of old and new machine learning technologies, to provide robust solutions to big problems within the machine learning and machine intelligence communities;

In order to create AI+ we need to build intelligent systems that leverage new and existing techniques, to generate informed, rational decisions in an automated way.To do this we will employ probabilistic programming (PP) Gordon et al.

On the other hand, the statistics and machine learning community, who look at how one can apply PPLs to the real world, via flexible systems that leverage existing inference algorithms Hoffman et al.

It is common among those of the latter vantage point to call a PPS a PPL, throughout this work I shall keep the distinction clear.  The need for large-scale manual annotations is a bottleneck for many machine learning methods that use deep neural networks, especially for computer vision problems such as image classification.

Consequently stationary distributions must be simulated for many real-world applications in computer vision and reinforcement learning, where for example video and game sequence data are both highly temporally correlated, meaning online (real time) learning and testing is inhibited.

the ability for humans to immediately retain specific observed events) is also hindered by catastrophic interference, as higher retention of new function modes equates to faster catastrophic forgetting of old ones.

The understanding that we develop from observing a large amount of sensory information (often at young ages) facilitate us to achieve two tasks: 1) to predict certain properties of unseen objects (e.g.

For embodied systems not only is there a cost (either monetary or execution time) associated with an episode, thereby limiting the number of training samples obtainable, but there also exist safety constraints making exploration of state space undesirable.

Deep generative neural networks and their conditional variants have recently witnessed a surge of interest due to their impressive ability to model very complex probability distributions, such as the modelling of human face images or human voice audio signal.

A drone with these capabilities would be able to provide an autonomous aerial inspection of a demarcated area, navigating the environment and computing a complete dense reconstruction in a closed loop.

For instance, we seek to build on recently proposed methods based on piecewise-deterministic Markov processes — such as the bouncy particle sampler — which have demonstrated scalability by providing a mechanism to subsample data correctly.

Reinforcement learning (RL) aims to train systems that choose optimal actions given the state of their environment, by allowing agents to explore possible policies and learn from their experiences.

Our aim is, starting with an incomplete and uncertain model of the system dynamics, to design a controller which: ML approaches, for the most part, have been concerned with finding optimal policies and not guarantees about properties of the system and its behaviour while training and in operation.

On the other hand, system verification and robust control theory usually deal with the model uncertainty by establishing desirable system properties and investigating whether a system respects them, but with less focus on performance.

Judea Pearl: Causal Reasoning, Counterfactuals, Bayesian Networks, and the Path to AGI | AI Podcast

Judea Pearl is a professor at UCLA and a winner of the Turing Award, that's generally recognized as the Nobel Prize of computing. He is one of the seminal ...

Latent Variables: Bayesian Mixed Graph Models in Supervised and Unsupervised Learning

Hidden common causes are often the explanation behind the observed association of our recorded variables. In many cases, however, we are interested in ...

Abductive reasoning

Abductive reasoning (also called abduction, abductive inference or retroduction) is a form of logical inference that goes from an observation to a hypothesis that ...

Naive Bayes Classification with R | Example with Steps

Provides steps for applying Naive Bayes Classification with R. Data: R file: Machine Learning videos: .

Developing Bug-Free Machine Learning Systems Using Formal Mathematics

Noisy data, non-convex objectives, model misspecification, and numerical instability can all cause undesired behaviors in machine learning systems. As a result ...

Stochastic gradient descent

Stochastic gradient descent is a gradient descent optimization method for minimizing an objective function that is written as a sum of differentiable functions.

Evolutionary AI: A Better Decision-Making Strategy for Business | Cognizant

Often when we talk to businesses, what their expectation from AI is more insights. Whereas AI can help with the actual decision-making too. If you actually have ...

Decision Tree In R | Decision Tree Algorithm | Data Science Tutorial | Machine Learning |Simplilearn

This Decision Tree in R tutorial video will help you understand what is decision tree, what problems can be solved using decision trees, how does a decision ...

6/6/14 Development of Intelligence - Laura Schulz: Inference and Exploration in Early Childhood