人工智能研究院学术报告 第2023
报告题目InnovatingandInterpretingNeuralNetworks
报告摘要Deeplearninghasrecentlyachievedhugesuccessinmanyapplications,includingnaturallanguageprocessing,computervisionandmore.Inthesecases,deeplearningcanoutperformorcompetewithhumans.Itiswidelyrecognizedthatmachinelearning,especiallydeeplearning,isaparadigmshiftinmanyfields.However,therearestillmanychallengesahead.Ononehand,overthepastyears,majoreffortshavebeendedicatedtoarchitectureinnovationsinthefieldofneuralnetworks,leadingtomanyadvancedmodels.Althoughdeeplearningisinspiredbythecomputationoftheneuralsystem,currentdeeplearningsystemsfallshortofreflectingneuraldiversity.Ontheotherhand,despitethefactthatdeeplearningperformsquitewellinpractice,itisdifficulttoexplainitsunderlyingmechanismandunderstanditsbehaviors.Thesuccessofdeeplearningisnotwellunderpinnedbytheeffectivetheory.Lackinginterpretabilityhasbecomeaprimaryobstacletothewidespreadtranslationandfurtherdevelopmentofdeeplearningtechniques.Inthisproject,weproposequadraticneuronstoaddresstheneuraldiversityproblemindeeplearning,whereinnerproducts(whicharelinearoperations)arereplacedwithquadraticcounterpartswhosenon-linearityenhancestheexpressiveabilityoftheneuron.Further,weproposesoftthresholdingtoreplaceReLUactivationforsignalprocessingtasks.Wewillevaluatetheirfeasibilityinpracticalcomputervisionproblemsaswellasmedicalimagingproblems,therebyenrichingmachinelearningarmory.Wewillalsodevelopinterpretationmethodsfortheinnerworkingofneuralnetworksandaccountabletheoriesforthesuccessofdeepnetworks.