Charles, your article provides a compelling argument against directly incorporating domain knowledge into model architectures. I'm curious to hear your thoughts on approaches like Retrieval Augmented Generation (RAG), which aim to leverage external knowledge sources during model inference. Do you see RAG as a potential solution to some of the challenges you've outlined, or do you believe that the risks associated with incorporating domain knowledge still outweigh the benefits? Looking forward to your insights!
Charles, your article provides a compelling argument against directly incorporating domain knowledge into model architectures. I'm curious to hear your thoughts on approaches like Retrieval Augmented Generation (RAG), which aim to leverage external knowledge sources during model inference. Do you see RAG as a potential solution to some of the challenges you've outlined, or do you believe that the risks associated with incorporating domain knowledge still outweigh the benefits? Looking forward to your insights!
What ways can bias be quantified in the architecture or training dataset from the resulting model?
Probably a better approach unless the prior knowledge is explicitly calculable like a an equation you want to minimize a loss function for.