PhD position - Training and quantization of large-scale deep neural networks for transfer learning

OPPORTUNITY DETAILS

Total reward

0 $

State University

CEA Tech

Area

Western Europe

Host Country

France

Deadline

31 Oct 2021

Study level

Doctoral Degree

Opportunity type

PhD

Specialities

Information technology and computer science

Opportunity funding

Not funding

Eligible Countries

This opportunity is destined for all countries

Eligible Region

All Regions

SL-DRT-21-0446

RESEARCH FIELD

Artificial intelligence & Data intelligence

ABSTRACT

Training and quantization of large-scale deep neural networks for transfer learningTransfer learning is today a common technique in Deep Learning that uses the learned parameters of a generic network (a feature extractor) to accelerate the training of another network on a more specific task. This specialized network is subsequently optimized for the hardware constraints of the specific use-case. However, given that the representations of the feature extractor are often rather generic, it might be possible to optimize the parameters before the transfer, to avoid that each end-user has to perform this optimization by herself. In this context, the thesis has the following scientific objectives:- Using several “unsupervised” learning methods (self-supervised, weakly supervised, semi-supervised) to train feature extractors on large datasets- Studying how common optimization methods (in particular quantization) can be applied on these extractors in a “task-agnostic” fashion- Quantifying the influence of these optimizations on the transfer learning capacity, by benchmarking and theoretical analysis (e.g. information compression theory)Required competences: Master degree (or equivalent), machine learning (in particular Deep Learning), programming (Python, Pytorch, Tensorflow, C++), good English (French knowledge is not required, but helpful)