SmartPrompt: Self-Learning Prompt Optimization in Generative AI Using Reinforcement Learning and Diffusion Models
Keywords:
SmartPrompt, reinforcement learning, diffusion models, large language models, coherenceAbstract
Self-learning prompt optimization in large language models (LLMs) is done by integration of reinforcement learning (RL) with diffusion-based generative models which is introduced as an advanced framework called SmartPrompt. The objective of this paper is to dive deep into the proposed methodology that refines input prompts dynamically through policy gradient updates in RL and optimize them for coherence, factual accuracy, and contextual appropriateness.
Downloads
References
A. Radford, L. Narasimhan, T. Salimans, and I. Sutskever, "Learning to Generate Reviews and Discovering Sentiment," arXiv preprint arXiv:1704.01444, 2017.
R. K. Gupta, R. K. Prasad, and D. K. Soni, "Deep Reinforcement Learning in Natural Language Processing: A Review," Journal of Artificial Intelligence Research, vol. 68, pp. 1-45, 2021.
T. B. Brown, M. Mann, N. Ryder, and others, "Language Models are Few-Shot Learners," arXiv preprint arXiv:2005.14165, 2020.
Y. Kim, "Convolutional Neural Networks for Sentence Classification," Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1746-1751, 2014.
P. Vaswani, A. Shazeer, N. Parmar, and others, "Attention is All You Need," Proceedings of NeurIPS, vol. 30, 2017.
H. V. Hasselt, "Double Q-learning," Proceedings of NeurIPS, vol. 24, 2011.
X. Chen, J. Song, X. Li, and others, "Diffusion Models Beat GANs on Image Synthesis," arXiv preprint arXiv:2105.05233, 2021.
I. Goodfellow, J. Pouget-Abadie, M. Mirza, and others, "Generative Adversarial Nets," Proceedings of NeurIPS, vol. 27, pp. 2672-2680, 2014.
A. Graves, A. Mohamed, and G. Hinton, "Speech Recognition with Deep Recurrent Neural Networks," IEEE Transactions on Audio, Speech, and Language Processing, vol. 61, no. 5, pp. 1049-1061, 2013.
A. Radford, J. Wu, D. Amodei, and others, "Learning Transferable Visual Models From Natural Language Supervision," Proceedings of NeurIPS, vol. 32, 2019.
J. Schulman, P. Abbeel, and X. Chen, "Trust Region Policy Optimization," Proceedings of ICML, pp. 1889-1897, 2015.
T. Salimans, I. Goodfellow, W. Zaremba, and others, "Improved Techniques for Training GANs," Proceedings of NeurIPS, vol. 29, 2016.
L. Chen, Z. Zeng, and Q. Xie, "Generative Diffusion Models for Neural Machine Translation," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6197-6207, 2021.
R. J. Williams, "Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning," Machine Learning, vol. 8, no. 3, pp. 229-256, 1992.
M. L. S. Ruder, "An Overview of Multi-task Learning in Deep Neural Networks," Proceedings of the International Conference on Neural Information Processing Systems (NeurIPS), 2017.
J. D. Anderson, G. S. Rios, and T. S. Ma, "Reinforcement Learning for Natural Language Processing: A Survey," Journal of Machine Learning Research, vol. 20, pp. 1-56, 2019.
L. Kingma and D. P. Rezende, "Auto-Encoding Variational Bayes," Proceedings of the International Conference on Learning Representations (ICLR), 2014.
A. Dosovitskiy, J. T. Springenberg, and F. Riedmiller, "Discriminative Unsupervised Feature Learning with Exemplar Convolutional Neural Networks," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 9, pp. 1734-1747, 2016.
A. Radford, J. W. J. Jr., and I. Sutskever, "Learning to Generate Text: A Survey of Neural Language Models," Proceedings of the International Conference on Learning Representations (ICLR), 2017.
Z. Yang, C. D. Manning, and L. Xie, "Prompt Engineering for Open-Domain Conversational AI," Proceedings of NeurIPS, vol. 34, 2021.
