architecture. Instantiating a configuration with the defaults will yield a similar configuration to that ofRoBERTa has almost similar architecture as compare to BERT, but in order to improve the results on BERT architecture, the authors made some simple design changes in its architecture and training procedure. These changes are:Use it as a regular