Unfortunately, our Paper which we submitted to the WWW conference, was rejected:
The selection process was highly competitive; we received 658 submissions out of which 82 were accepted as regular papers and 29 were accepted as posters.
Overall, our idea seemed interesting to the reviewers, but the did not provide enough evidence to convince them:
=== Reviewer 1 ===
OVERALL RATING: -2 (Probable reject, unless someone champions)
REVIEWER’S CONFIDENCE: 3 (high)
What is the rank of this submission in your pile?: 5 (5)
Is the submission worth considering for the Posters track?: 2 (yes)The paper describes a novel workflow management system for software engineering projects. This system is based on Twitter as a communication backbone. One of the nice features of this system is that it integrates human and machine work, thereby adopting a crowdsourcing paradigm.
I really wanted to like this paper. It is well written (mostly) and it is somewhat cool by leveraging crowdsourcing and twitter to solve technical (software engineering) problems. At the end, however, I just could not convince myself that there was significant technical content here. It just felt like building a big system without much behind it. If people want to use twitter to coordinate their (distributed) software engineering tasks, why don’t they just do it? Why do they need Tweetflow and the whole mechanism that it involves? I just did not understand what the big innovation in this paper is and how it was applied to solve a really interesting problem.
a.) Motivating Scenario: It is a cool project for a class on software engineering. But, do you really need to write about this in a top-tier conference like WWW?
b.) Twitter: Twitter was described like a holy grail that solves all problems. Actually for this particular scenario it adds some problems because twitter data is open on the web. I can imagine that many software projects would require more privacy. So, why don’t you just build your own twitter? That way, a great deal of the limitations of Twitter that you seem to be fighting in the paper go away, too. Building Twitter is not really rocket science. (Maybe, building Twitter at Web-scale is rocket science, but you do not really need that for your purposes.)
c.) Why is Twitter a better tool for service discovery than the Web-based interface of microtask platforms such as Mechanical Turk. I just did not see the magic here.
d.) In 3.2, the paper mentions the difficulty of human/machine communication. There is a whole community that deal with that issue (HCI). The paper does not add much to that topic. In particular, the language proposed did not add much to that – might be okay for people who are used to tweak messages into Twitter, but not for everybody.
e.) It would have been interesting to read about some of the experiences made with the technology at work. Architecture etc. are pretty straightforward. The game-changer of crowdsourcing is to make it really happen.
=== Reviewer 2 ===
OVERALL RATING: -2 (Probable reject, unless someone champions)
REVIEWER’S CONFIDENCE: 3 (high)
What is the rank of this submission in your pile?: 2 (2)
Is the submission worth considering for the Posters track?: 2 (yes)This paper presents a framework for offering and consuming services, mainly human offered services, on Twitter. In a way it provides an implementation of MechanicalTurk over tweets for finding relevant service providers. The paper is timely, and on an interesting topic. The paper is not very well written, but still readable. It requires some efforts to understand the paper.
However, the paper does not offer what it claims (in the title and abstract), and still the approach is in an early development stage. In particular, how workflows are formed, supported and monitored, and how Web services are found, consumed and monitored are never explained in a clear manner in the paper. I would rather be happier if the paper claims to provide a framework for offering and consuming services (mainly human provided) over Twitter. However, still in this sense it is not clear from the paper, how automated services would be able to accept and process tweets (as they are mainly text, unless the services that are designed to so so and receive a strictly formatted text as input).
The cryptic language that is presented for Twitter is still complex for human users to parse and comprehend. This is particularly challenging due to the limit on the number of tweet characters.
Although, there is a discussion on the architecture of the solution, its implementation and whether a prototype has been built and evaluated is not discussed in the paper. Nevertheless, experience on using the current Twitter site for some initial experiments on using this approach is reported among a group of students and researcher.
Given the approach presented in this paper is interesting, and as an application is novel, but in its very early stages of development, I suggest it to be referred to poster track for consideration.
minor comments:
- page 3, right column, paragraph before section 3.2, “can be discovery” => “can be discovered”
- page 3, right column, paragraph before section 3.2, “to discovery” => “to discover”
- the first sentence in section 3.2, please revise, not complete sentence (in terms of meaning)
- section 3.2, lines -3 and -3 in page 3, please revise “we strive … which are descriptions…” meaning not clear
- page 4, the few sentences before 3.3., “After … workflow” is long and not clear, what do you mean? consider breaking into smaller sentences
=== Reviewer 3 ===
OVERALL RATING: -2 (Probable reject, unless someone champions)
REVIEWER’S CONFIDENCE: 4 (expert)
What is the rank of this submission in your pile?: 2 (2)
Is the submission worth considering for the Posters track?: 2 (yes)Authors propose use of Twitter as a orchestration platform between human and software services in the context of
workflows. The overarching concept is integration of human driven services in a service-oriented manner, following SOA principles. Authors propose dynamic embodiments of services at any point in time of workflow execution. By monitoring traces of social interactions authors propose to monitor tasks progress thereby making it transparent. The idea is interesting, however there is no substantial background about the implementation details, and the value of this technical approach. It is neat – by why is it significant? Furthermore, the paper could be significantly improved by including more evaluation of the Tweetflows.. e.g. scalability, etc.
This comments help us to improve our work and we thank the reviewers for their work. We already started to use Tweetflows with student projects and hope to gain interesting insights from this experiment.
your ikangai science team
How difficult can it be to write a Twitter iPhone Client? Not very difficult, if you have help from various sources. All you need is to add some source files into your XCode Twitter Client project and that’s it. Well, actually, it’s not THAT easy. You need some additional steps to do. An excellent tutorial that guides you through this can be found