2 PhD research fellowship positions within “Explainable AI systems for business-critical... (# of pos: 2)
NTNU Norwegian University of Science and Technology

2 PhD research fellowship positions within “Explainable AI systems for business-critical... (# of pos: 2)

Norway 15 Feb 2021


NTNU Norwegian University of Science and Technology
Visit institution page


State University
Host Country
15 Feb 2021
Study level
Opportunity type
Eligible Countries
This opportunity is destined for all countries
Eligible Region
All Regions

This is NTNU

At NTNU, creating knowledge for a better world is the vision that unites our 7 400 employees and 42 000 students.

We are looking for dedicated employees to join us.

You will find more information about working at NTNU and the application process here.

About the position

Explainable AI and the EXAIGON project

The recent rapid advances of Artificial Intelligence (AI) hold promise for multiple benefits to society in the near future. AI systems are becoming ubiquitous and disruptive to industries such as healthcare, transportation, manufacturing, robotics, retail, banking, and energy. According to a recent European study, AI could contribute up to EUR 13.3 trillion to the global economy by 2030; EUR 5.6 trillion from increased productivity and EUR 7.73 trillion from opportunities related to consumer experience. However, in order to make AI systems deployable in social environments, industry and business-critical applications, several challenges related to their trustworthiness must be addressed first. 

Most of the recent AI breakthroughs can be attributed to the subfield of Deep Learning (DL), based on Deep Neural Networks (DNNs), which has been fueled by the availability of high computing power and large datasets. Deep learning has received tremendous attention due to its state-of-the-art, or even superhuman, performance in tasks where humans were considered far superior to machines, including computer vision, natural language processing, and so on. Since 2013, Deep Mind has combined the power of DL with Reinforcement Learning (RL) to develop algorithms capable of learning how to play Atari games from pixels, beating human champions at the game of Go, surpassing all previous approaches in chess, and learning how to accomplish complex robotic tasks. Similarly, DL technology has been used in combination with Bayesian Networks (BNs), resulting in Deep Bayesian Networks (DBNs), a framework that dramatically increases the usefulness of probabilistic machine learning. Despite their impressive performance, DL models have drawbacks, with some of the most important being lack of transparency and interpretability, lack of robustness, and inability to generalize to situations beyond their past experiences. These are difficult to tackle due to the black-box nature of DNNs, which often end up having millions of parameters, hence making the reasoning behind their predictions incomprehensible even to human experts. In addition, there is a need to better understand societal expectations and what elements are needed to ensure societal acceptance of these technologies.

Explainable AI (XAI) aims at remedying these problems by developing methods for understanding how black-box models make their predictions and what are their limitations. The call for such solutions comes from the research community, the industry and high-level policy makers, who are concerned about the impact of deploying AI systems to the real world in terms of efficiency, safety, and respect for human rights. In order for XAI to be useful in business-critical environments and applications, it should not be limited to algorithm design because the experts who understand decision-making models the best are not in the right position to judge the usefulness and structure of explanations. It is necessary to enhance XAI research by incorporating models of how people understand explanations, and when explanations are sufficient for trusting something or someone. Such models have been researched for many years by philosophers, social and cognitive psychologists, and cognitive scientists. It becomes evident that significant interdisciplinary contributions are needed for AI systems to be considered trustworthy enough for deployment in social environments and business-critical applications.

The EXAIGON (Explainable AI systems for gradual industry adoption) project (2020-2024) will deliver research and competence building on XAI, including algorithm design and human-machine co-behaviour, to meet the society’s and industry’s standards for deployment of trustworthy AI systems in social environments and business-critical applications. Extracting explanations from black-box models will enable model verification, model improvement, learning from the model, and compliance to legislation. 

EXAIGON aims at creating an XAI ecosystem around the Norwegian Open AI-Lab, including researchers with diverse background and strong links to the industry. The project is supported by 7 key industry players in Norway who will provide the researchers with use cases, including data, models and expert knowledge. All involved researchers will work closely with each other, the industry partners, and researchers already working on relevant topics at NTNU, hence maximizing the project’s impact and relevance to the real world.  

You will report to the Head of the Department.

Duties of the position

EXAIGON is hiring 2 PhD candidates, one in each of the following areas:

PhD1: XAI methods for deep Bayesian networks. Probabilistic AI investigates the intersection between probabilistic graphical models and deep learning. Such models are often called deep Bayesian networks (DBNs). Injecting probabilistic thinking into DNNs has several benefits, including robustness against overfitting and resilience against adversarial attacks. Furthermore, DBNs can quantify uncertainty in their predictions, also in ways that include model and parameter uncertainty. DBNs will to some extent cater to a causal interpretation, which provides an efficient and robust language for explaining inferences. In EXAIGON we aim to utilize these features in order to generate understandable and trustworthy explanations for model-predictions from probabilistic AI models. Furthermore, we will investigate how techniques for explanations and sensitivity analysis used for traditional Bayesian networks carry over to Probabilistic AI models. We will first consider general strategies for generating explanations from DBNs, and later apply the most promising techniques in industrial settings.

For this position, the selection committee will prioritize candidates with a strong knowledge of mathematical statistics, combined with theoretical and hands-on experience with modern machine learning. The following qualifications are required:

  • Documented strong background in Machine Learning, with formal training in mathematical statistics and/or computer science.
  •  A general knowledge of the trends in Probabilistic AI and deep learning.
  • Excellent programming skills in Python.

The following qualifications will be considered as an advantage:

  • Experience with deep learning frameworks like Tensorflow and PyTorch.
  • To communicate with external project partners, excellent written and oral Norwegian language skills are a plus.
  • Research-oriented master thesis within the area of Artificial Intelligence.
  •  Ability to work independently as well as collaboratively.

PhD2: XAI methods using Case-Based Reasoning. Case-Based Reasoning (CBR) can take advantage of human experiences in decision making by incorporating those experiences. This capability can be used to build explainable CBR systems (model-specific) as well as explaining other AI system (model-agnostic). Both approaches will be researched in the position with a focus on creating explainable CBR systems. It is expected that the successful candidate will build prototypes for various approaches and evaluates them with the collaborators in the project. For this position, the selection committee will prioritize candidates with previous knowledge in Case-Based Reasoning and a strong knowledge of theoretical and hands-on experience with modern machine learning. It is expected that the successful candidate is a skilled Python programmer, and experience with Java software development is an advantage.

For this position, the candidate should have the following qualifications:

  • Strong background in one or more of the following areas: Artificial Intelligence or Machine Learning.
  • Knowledge (practical or theoretical) of Case-Based Reasoning.
  • Excellent programming skills - in one or more of the following languages: Python, Java.
  • Knowledge of current Machine Learning frameworks such as scikit-learn, tensorflow, etc.

The following qualifications will be considered as an advantage:

  • To communicate with external project partners, excellent written and oral Norwegian language skills are a plus.
  •  Research-oriented master thesis within the area of Artificial Intelligence.
  • Knowledge of neural network architecture and applications.
  • Ability to work independently as well as collaboratively.

The candidates will have the opportunity to combine experimental and theoretical work in the aforementioned areas, with the aim of eventually demonstrating their successful fusion in realistic conditions. All PhD candidates will also have the option for a research stay abroad during their studies. Potential destinations include USA and Australia, where EXAIGON has established scientific collaborators.  

Required selection criteria

We seek two ambitious and highly motivated individuals with a Master’s degree in engineering cybernetics, computer science, control engineering, applied mathematics, or a related discipline. Applicants are required to justify their candidateship by explicitly explaining their personal motivation and academic aptitude for pursuing a doctoral degree within this research field.

Applicants that expect to complete their master’s degree studies by summer 2021 can apply.

Academic results, publications, relevant specialization, work or research experience, personal qualifications, and motivation will be considered when evaluating the applicants.

It is a prerequisite that the PhD scholar applies for and is granted admission to the NTNU PhD studies as soon as possible after employment. NTNUs PhD-rules require a master’s degree or equivalent with at least 5 years of studies and an average ECTS grade of A or B within a scale of A-E for passing grades (A best). Applicants must be qualified for admission as PhD students at NTNU. See http://www.ime.ntnu.no/forskning/phd  for information about PhD studies at NTNU.

Excellent English skills, written and spoken, are required. Applicants from non-European countries where English is not the official language must present an official language test report. The following tests can be used as such documentation: TOEFL, IELTS or Cambridge Certificate in Advanced English (CAE) or Cambridge Certificate of Proficiency in English (CPE). Minimum scores are:

  • TOEFL: 600 (paper-based test), 92 (Internet-based test)
  • IELTS: 6.5, with no section lower than 5.5 (only Academic IELTS test accepted)
  • CAE/CPE: grade B or A.

Appointments are made in accordance with the regulations in force regarding terms of employment for PhD candidates issued by the Ministry of Education and Research, with relevant parts of the additional guidelines for appointment as a PhD candidate at NTNU.

Applicants must undertake to participate in an organized PhD programme of study during their period of employment. The person who is appointed must comply with the conditions that apply at any time to employees in the public sector. In addition, a contract will be signed regarding the period of employment.

The appointment is to be made in accordance with the regulations in force concerning State Employees and Civil Servants and national guidelines for appointment as PhD, post doctor and research assistant .

Personal characteristics

In the evaluation of which candidate is best qualified will be placed on education, experience and personal suitability.

We offer
Salary and conditions

PhD candidates are remunerated in code 1017, and are normally remunerated at gross from NOK 482 200 per annum before tax, depending on qualifications and seniority. From the salary, 2% is deducted as a contribution to the Norwegian Public Service Pension Fund.

The Faculty of Information Technology and Electrical Engineering wants to attract outstanding and creative candidates who can contribute to our ongoing research activities. We believe that diversity is important to achieve a good, inclusive working environment. We encourage all qualified candidates to apply, regardless of the gender, disability or cultural background. 

Under Section 25 of the Freedom of Information Act, information about the applicant may be made public even if the applicant has requested not to have his or her name entered on the list of applicants. 

The engagement is to be made in accordance with the regulations in force concerning State Employees and Civil Servants, and the acts relating to Control of the Export of Strategic Goods, Services and Technology. Candidates who by assessment of the application and attachment are seen to conflict with the criteria in the latter law will be prohibited from recruitment to NTNU. After the appointment you must assume that there may be changes in the area of work.

The period of employment is 3 years or 4 years with 25 % duty work.

The position is subject to external funding of “Explainable AI and the EXAIGON project”.

It is a prerequisite you can be present at and accessible to the institution on a daily basis.

About the application

The application and supporting documentation to be used as the basis for the assessment must be in English. 

Publications and other scientific work must follow the application. Please note that applications are only evaluated based on the information available on the application deadline. You should ensure that your application shows clearly how your skills and experience meet the criteria which are set out above. 

The application must include:

  • An explicit indication of which of the two positions (PhD1 or PhD2), the applicant applies for. If an applicant’s background and experience justify it, one can apply for both positions. In that case, the applicants’ research statement should reflect information pertaining both positions. 
  •  CV including information relevant for the qualifications and contact information for at least 2 reference persons.
  • Certified copies of academic diplomas and transcripts.
  • Applicants from universities outside of Norway are requested to send a diploma supplement (or a similar document) which describes in detail the study and grading system, and the rights for further studies associated with the obtained degree.
  • A research statement (max. 3 pages) including:
    •  A short presentation of the motivation for a PhD study. 
    •  Why the applicant is suited for the position.
    • The applicant’s view of research challenges for the PhD position, as well as his/her theoretical and methodological approach to the challenges.

Joint works will be considered. If it is difficult to identify your contribution to joint works, you must attach a brief description of your participation.

In the evaluation of which candidate is best qualified, emphasis will be placed on education, experience and personal suitability.

NTNU is committed to following evaluation criteria for research quality according to The San Francisco Declaration on Research Assessment - DORA.

General information

Working at NTNU

A good work environment is characterized by diversity. We encourage qualified candidates to apply, regardless of their gender, functional capacity or cultural background. 

The city of Trondheim is a modern European city with a rich cultural scene. Trondheim is the innovation capital of Norway with a population of 200,000. The Norwegian welfare state, including healthcare, schools, kindergartens and overall equality, is probably the best of its kind in the world. Professional subsidized day-care for children is easily available. Furthermore, Trondheim offers great opportunities for education (including international schools) and possibilities to enjoy nature, culture and family life and has low crime rates and clean air quality.

As an employee at NTNU, you must at all times adhere to the changes that the development in the subject entails and the organizational changes that are adopted.

Information Act (Offentleglova), your name, age, position and municipality may be made public even if you have requested not to have your name entered on the list of applicants.

Further details about the positions can be obtained from:

PhD1: Professor Helge Langseth, e-mail: helge.langseth@ntnu.no

PhD2: Associate Professor Kerstin Bach, e-mail: kerstin.bach@ntnu.no

Or the Head of the Department, Professor John Krogstie, e-mail: john. krogstie@ntnu.n

Please submit your application electronically via jobbnorge.no with your CV, diplomas and certificates. Applications submitted elsewhere will not be considered. Diploma Supplement is required to attach for European Master Diplomas outside Norway. Chinese applicants are required to provide confirmation of Master Diploma from China Credentials Verification (CHSI) .

Publications and any other work that the applicant wishes to be considered must also be enclosed.

If you are invited for interview you must include certified copies of transcripts and reference letters. Please refer to the application number 2020/24842 when applying.

Application deadline: 15.02.2021

NTNU - knowledge for a better world

NTNU - knowledge for a better world

The Norwegian University of Science and Technology (NTNU) creates knowledge for a better world and solutions that can change everyday life.

Department of Computer Science

We are the leading academic IT environment in Norway, and offer a wide range of theoretical and applied IT programmes of study at all levels. Our subject areas include hardware, algorithms, visual computing, AI, databases, software engineering, information systems, learning technology, HCI, CSCW, IT operations and applied data processing. The Department has groups in both Trondheim and Gjøvik. The Department of Computer Science  is one of seven departments in the Faculty of Information Technology and Electrical Engineering  .

Choose your study destination

Choose the country you wish to travel to study for free, work or volunteer

Please find also

Featured tags

phd scholarships 2021 PhD program PhD Theses