how to improve system using relevance feedback?

Hi, I just stuck here in my project..I am using algorithm Sift to retrieve similar images as that of query image.I am also using relevance feedback where the user gives its feedback in case negative images are present.I want to know how to make ur system more intelligent so that after 1 st feedback it will show those images that are really relevant without user intervention.Somewhere I read to update weights.

Thanks preeti