ICASSP 的comments

早先在ICASSP 就被罵過一次了....
呼, 這個題目看來機遇不佳,我還得再加把勁才行啊!!

---- Comments from the Reviewers: ----

While the paper presents an interesting strategy to perform information retrieval, I think that calling this strategy dialog management in the context of conversational applications seems to be a quite narrow interpretation of it. As a matter of fact, a "simulated user" rather that simulating conversational activity seems to focus on query terms. So, all in all, the combination of "simulated users for tunning dialog management" in reality, seems better described by query optimization via key term search.

-----
This paper describes a document retrieval process based on query refinement technique:
the documents to retrieve (the target) are grouped in a set D.
A first query, made of a single key term, leads to a first set of documents. By adding another key term precising the query, a smaller set of documents is obtained (the intersection of the documents obtained with both key terms) and this process goes on until the recall on the K first documents obtained with the multiple key term query is above a given threshold. This recall is estimated according to the target D.

I have several issues with this paper:
1) I believe this work is not relevant to the ICASSP conference, as there is no speech technology involve in the process described
2) I think the title is misleading: I don't see any dialogue strategy in this work. The process described sorts key terms in order to retrieve documents. In the experiments, a key term is randomly selected, then the next key term is chosen according to the tree, and if I understood correctly, the simulated user systematically accept this new key term. So where is the dialogue?
3) I have also some issue with the methodology chosen: how do you randomly chose the set of target document D? How can you control that this set of document can represent a "real" query that can be made by a user?
4) The baseline system is not clearly presented. Is the high failure rate due to a choice of key term that retrieve no documents of D? In this case this evaluation is very artificial, as why a real user would validate a key term outside the scope of his query ?



-----
This paper describes an approach for presenting the user with a hierarchical set of new keywords (which may be presented to the user in a limited space, such as the screen of a hand-held device) to narrow down the search space for document retrieval. The interaction going on between the user and the system is not exactly the type of the dialog that in general people are dealing with, but the application itself is very interesting. However, the testing and evaluation is not clearly defined in the paper, and leaves the author with many questions about the use of the approach. For example, how are the documents in D selected? How is it guaranteed that they are about a specific topic, which may be specified with a few key terms. Also, it would be useful if the authors relate the terms in the paper with the general reinforcement framework (e.g., feedback, etc).
What happens if the user wants to select another term not in the hierarchy?
What is r_0 (recall rate threshold) for the experiments (testing)? If it is also 0.1, isn’t it too low for testing?
What is the difference between a and b, or c and d in Figure 5? Why doesn’t c (a) start at the same point with d (b)?

Here are some typos:
- “ the system interactively help the user” -> “ the system interactively helps the user”
- “ In general all other key term…” -> : In general all other key terms”
- “Each of these possible key term is” –> “Each of these possible key terms are”
- “ all possible expansion” -> “all possible expansions”



-----