Seminar Web Engineering (WS 2023/2024)

Questions:

• What is the current state of voice user interface research? Identify relevant publication venues, classify/group existing published approaches and identify research directions for future research as well as tools and platforms that support the creation of voice user interfaces.
• Which approaches specifically address the automatic assessment, evaluation and testing of voice user interfaces or voice interactions? Which methods do they use? What kinds of inputs do they require? Which results/predictions/assessments do they produce?
• What are quality and performance metrics for voice user interfaces? Identify existing measurement strategies and metrics that allow the evaluation and comparison of voice user interfaces.

Literature:

• https://www.sciencedirect.com/science/article/pii/S2667295222000216
• https://stccj.ucv.ro/index.php/stccj/article/view/26
• https://dl.acm.org/doi/abs/10.1145/3313831.3376522
• https://dl.acm.org/doi/abs/10.1145/3173574.3174214
• https://dl.acm.org/doi/abs/10.1145/3296699

Questions:

• What is the difference between citation, bibliography and bibliometrics? What is their importance for scientific works?
• Explain and differentiate at least 5 common citation styles for computer science (IEEE, APA, ACM, ...). Show this with examples.
• What are rules and expectations for a bibliography?
• Which tools/programms can be used for citations and bibliography while writing a paper?
• What metrics can be used for bibliography? What are its differences? Where are their limits? Show these on examples.

Literature:

• Own research
• https://www.tu-chemnitz.de/ub/publizieren/bibliometrie/indikatoren.html
• Hicks, D., Wouters, P., Waltman, L. et al. Bibliometrics: The Leiden Manifesto for research metrics. Nature 520, 429–431 (2015). https://doi.org/10.1038/520429a
• Bornmann L, Mutz R, Neuhaus C, Daniel HD (2008) Citation counts for research evaluation: standards of good practice for analyzing bibliometric data and presenting and interpreting results. Ethics Sci Environ Polit 8:93-102. https://doi.org/10.3354/esep00084
• https://www.boisestate.edu/cobe/cobe-writing-style-guide/citations-and-references/#:~:text=Citations%20include%20the%20author%27s%20last,%2C%20if%20it%20is%20lengthy
• https://www.acm.org/publications/authors/reference-formatting
• https://ieee-dataport.org/sites/default/files/analysis/27/IEEE%20Citation%20Guidelines.pdf
• https://apastyle.apa.org/
• http://www.metrics-toolkit.org/

Questions:

• Identify approaches published in scientific literature and existing tools/frameworks that make use of WebAssembly for Code Mobility, to execute code written in one single language on server and client side, or outside the browser and briefly describe them.
• Try to identify groups of approaches with similar architecture/purpose/technology.
• Prepare a demo applying at least one of the approaches to a scenario application to showcase the potential benefits.

Literature:

• Mäkitalo, N., Mikkonen, T., Pautasso, C., Bankowski, V., Daubaris, P., Mikkola, R., Beletski, O.: WebAssembly Modules as Lightweight Containers for Liquid IoT Applications. In: Proc. of ICWE2021. pp. 328–336. Springer, Cham (2021).
• Wen, Elliott, and Gerald Weber. "Wasmachine: Bring iot up to speed with a webassembly os." 2020 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops). IEEE, 2020.
• Ménétrey, Jämes, et al. "WebAssembly as a Common Layer for the Cloud-edge Continuum." Proceedings of the 2nd Workshop on Flexible Resource and Application Management on the Edge. 2022.
• Koren, István. "A standalone webassembly development environment for the internet of things." Web Engineering: 21st International Conference, ICWE 2021, Biarritz, France, May 18–21, 2021, Proceedings. Cham: Springer International Publishing, 2021.
• Hoque, Mohammed Nurul, and Khaled A. Harras. "WebAssembly for Edge Computing: Potential and Challenges." IEEE Communications Standards Magazine 6.4 (2022): 68-73.
• WASI https://wasi.dev/
• .NET Blazor
• https://www.thinktecture.com/blazor/unterschiede-blazor-webassembly-blazor-server/

Questions:

• What is the objective of query expansion? How does it related to query completion?
• Which approaches for query expansion exist?
• Which peculiarities must be taken into account for the use case of research data repositories?
• Demo: Choose a query expansion approach and demonstrate it. Use the RDA's search and interaction log (Wu & Benn) as a basis.

Literature:

• H. K. Azad and A. Deepak, “A novel model for query expansion using pseudo-relevant web knowledge,” Pattern Recognition Letters, vol. 158, pp. 148–156, Jun. 2022, doi: 10.1016/j.patrec.2022.04.013.
• M. A. Raza, R. Mokhtar, and N. Ahmad, “A survey of statistical approaches for query expansion,” Knowl Inf Syst, vol. 61, no. 1, pp. 1–25, Oct. 2019, doi: 10.1007/s10115-018-1269-8.
• J. A. Nasir, I. Varlamis, and S. Ishfaq, “A knowledge-based semantic framework for query expansion,” Information Processing & Management, vol. 56, no. 5, pp. 1605–1617, Sep. 2019, doi: 10.1016/j.ipm.2019.04.007.
• H. K. Azad and A. Deepak, “Query expansion techniques for information retrieval: A survey,” Information Processing & Management, vol. 56, no. 5, pp. 1698–1735, Sep. 2019, doi: 10.1016/j.ipm.2019.05.009.
• R. Stojanov, S. Gramatikov, I. Mishkovski, and D. Trajanov, “Linked Data Authorization Platform,” IEEE Access, vol. 6, pp. 1189–1213, 2018, doi: 10.1109/ACCESS.2017.2778029.
• J. Singh, A. Sharan, and S. Siddiqi, “A Literature Survey on Automatic Query Expansion for Effective Retrieval Task,” International Journal of Advanced Computer Research, vol. 3, Sep. 2013.
• C. Carpineto and G. Romano, “A Survey of Automatic Query Expansion in Information Retrieval,” ACM Comput. Surv., vol. 44, no. 1, p. 1:1-1:50, Jan. 2012, doi: 10.1145/2071389.2071390.
• X. Xu, F. Zhang, and Z. Niu, “An Ontology-Based Query System for Digital Libraries,” in 2008 IEEE Pacific-Asia Workshop on Computational Intelligence and Industrial Application, Dec. 2008, pp. 222–226. doi: 10.1109/PACIIA.2008.360.
• R. Sharifpour, M. Wu, and X. Zhang, “Large-scale analysis of query logs to profile users for dataset search,” JD, vol. 79, no. 1, pp. 66–85, Jan. 2023, doi: 10.1108/JD-12-2021-0245.
• Wu, Mingfang and Benn, Joel, “2019 search and interaction log from the data catalogue: Research Data Australia”. Zenodo, Feb. 20, 2022. doi: 10.5281/zenodo.6133000.
• Own research

Questions:

• What is Trust-aware Decision Making?
• What are common ways to create a trust-aware decision?
• In a scenario of dynamically acquiring data from decentralized knowledge graphs via the web, how could the scenario influence Trust-aware Decision Making?

Literature:

• https://vsr.informatik.tu-chemnitz.de/projects/2020/atlas/
• https://doi.org/10.1109/ACCESS.2013.2259892
• https://doi.org/10.1145/2816826

Questions:

• What is the approach of WebTransport?
• How is WebTransport different/new to common streaming technologies on the web like WebRTC?
• How well is WebTransport support by today and what are open challenges/discussions?

Literature:

• https://www.w3.org/TR/webtransport/#web-transport

Questions:

• What are UI elements? What are classical ones from graphical user interfaces (GUIs)? Which ones are used in conversational user interfaces (CUIs) and specifically chatbots? What is important about having good chatbot UI elements? Conduct a literature research on related work in UI research.
• Show good and bad examples of single chatbot features by using both the literature and searching for real-life examples on your own. Make a guideline on how to use chatbot UI elements in a good way.
• Show in the demonstration with a prototype (figma, etc.) which UI elements can be used for different use cases in chatbots. Compare the use of these UI elements with equivalent GUI elements and how they are used.

Literature:

• Own research
• Lukas A. Flohr, Sofie Kalinke, Antonio Krüger, and Dieter P. Wallach. 2021. Chat or Tap? – Comparing Chatbots with ‘Classic’ Graphical User Interfaces for Mobile Interaction with Autonomous Mobility-on-Demand Systems. In Proceedings of the 23rd International Conference on Mobile Human-Computer Interaction (MobileHCI '21). Association for Computing Machinery, New York, NY, USA, Article 21, 1–13. https://doi.org/10.1145/3447526.3472036
• Nguyen, Quynh N., Anna Sidorova, and Russell Torres. "User interactions with chatbot interfaces vs. Menu-based interfaces: An empirical study." Computers in Human Behavior 128 (2022): 107093. https://doi.org/10.1016/j.chb.2021.107093 (available via advisor)
• Khan, R., Das, A. (2018). Introduction to Chatbots. In: Build Better Chatbots. Apress, Berkeley, CA. https://doi.org/10.1007/978-1-4842-3111-1_1
• Mohit Jain, Pratyush Kumar, Ramachandra Kota, and Shwetak N. Patel. 2018. Evaluating and Informing the Design of Chatbots. In Proceedings of the 2018 Designing Interactive Systems Conference (DIS '18). Association for Computing Machinery, New York, NY, USA, 895–906. https://doi.org/10.1145/3196709.3196735
• https://www.tidio.com/blog/chatbot-ui/
• https://www.nngroup.com/articles/chatbots/

Questions:

• How can AI tools be used to help in the designing of User Interfaces? Split this up in different phases of IDEOs Design Thinking Process.
• Which AI tools are available? List them and build a list of suitable characteristics to compare them.
• Write frontend code for a simple use case (should include at least text, graphics, listings and two form inputs), this can include stock photos and lorem ipsum text. the code has to adher to common standards. Use at least two AI tool from your previous research which can be used to produce frontend code for the same use case and use them for the same layout.
• Compare your code the code from the AI tools and evaluate it. Keep in mind that accessibility of the frontend is expected according to the WCAG standard. What are benefits and drawbacks of using AI tools?

Literature:

• own research
• https://www.ideou.com/blogs/inspiration/what-is-design-thinking
• M. SureshKumar, G. Annamalai, S. K. Kumar and N. Susila, "AI Based UI Development Using Deep Neural Networks (AI-UI)," 2022 1st International Conference on Computational Science and Technology (ICCST), CHENNAI, India, 2022, pp. 387-391, doi: 10.1109/ICCST55948.2022.10040459.
• Yuwen Lu, Chengzhi Zhang, Iris Zhang, and Toby Jia-Jun Li. 2022. Bridging the Gap Between UX Practitioners’ Work Practices and AI-Enabled Design Support Tools. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems (CHI EA '22). Association for Computing Machinery, New York, NY, USA, Article 268, 1–7. https://doi.org/10.1145/3491101.3519809
• https://medium.com/@pbansiya/the-role-of-ai-in-designing-accessible-ui-ux-experiences-1495027593db
• https://bootcamp.uxdesign.cc/accessibility-testing-ui-with-ai-5a313d5f9e4a
• https://www.w3.org/TR/wcag-3.0/

Questions:

• What is Low-Code / No-Code? What is the idea behind it? What are Low Code Development Platforms?
• For which query languages has it been applied successfuly?
• What are the specifics of implementing an LCNC approach for query languages?
• Which challenges and limitations exist to the Low-Code / No-Code approach?
• How can the approach help in the context of Digital Transformation?

Literature:

• D. Di Ruscio, D. Kolovos, J. de Lara, A. Pierantonio, M. Tisi, and M. Wimmer, “Low-code development and model-driven engineering: Two sides of the same coin?,” Softw Syst Model, vol. 21, no. 2, pp. 437–446, Apr. 2022, doi: 10.1007/s10270-021-00970-2.
• V. S. Phalake and S. D. Joshi, “Low Code Development Platform for Digital Transformation,” in Information and Communication Technology for Competitive Strategies (ICTCS 2020), M. S. Kaiser, J. Xie, and V. S. Rathore, Eds., in Lecture Notes in Networks and Systems. Singapore: Springer Nature, 2021, pp. 689–697. doi: 10.1007/978-981-16-0882-7_61.
• A. Sahay, A. Indamutsa, D. Di Ruscio, and A. Pierantonio, “Supporting the understanding and comparison of low-code development platforms,” in 2020 46th Euromicro Conference on Software Engineering and Advanced Applications (SEAA), Portoroz, Slovenia: IEEE, Aug. 2020, pp. 171–178. doi: 10.1109/SEAA51224.2020.00036.
• N. Prinz, C. Rentrop, and M. Huber, “Low-Code Development Platforms – A Literature Review”.
• Own research

Questions:

• What is the current state of Web Engineering research? To answer this question, systematically analyze all publications of the 2 venues listed under literature as detailed below. Your primary information used should be the title, authors/affiliations, keywords, and abstract.
• For each publication, capture title, authors/affiliations, keywords, and abstract, venue, year, (for conference papers) name of track/workshop, (for journal articles) volume number, (for journal articles) issue number, (for journal articles) name of issue, page numbers in proceedings/issue, length of the publication, current number of citations of the publication, URL of online resource.
• Based on your raw data collection, analyze the following aspects: 1. What are the main topics of research interest and in which areas of the Web Engineering field, along with the number of publications belonging to them? 2. What authors are publishing in these venues, from which affiliations, from which countries, along with the number of publications for each of these? 3. Which are the most cited articles (relative to their age), which topics/areas receive the most citations, which authors/affiliations/countries receive the most citations? 4. Considering the time dimension, are there any visible trends for aspects 1-3 over the 5 years considered?
• Visualize your data and insights and provide the raw data in re-usable form (CSV).

Literature:

• Venue 1: ICWE Proceedings of last 5 complete years (2019-2023)
• Venue 2: JWE Journals Issues of last 5 complete years (2019-2023)
• For citation counts use: Google Scholar
• Tool for analysis and inspiration for your data visualization: https://www.connectedpapers.com/

Questions:

• Intro: Semantic Knowledge Graphs (KGs) and Large Language Models (LLMs) are two most prominent and powerful approaches to knowledge representation and information extraction with immense impact in a wide field of applications. A current debate focuses on the idea of unifying both LLMs and KGs, to leverage their respective strengths and overcome the limitations of each approach, for various downstream tasks.
• - Give a short introduction into the general approach and the state-of-the-art for KGs and LLMs
• - State and compare the respective strengths and weaknesses of KGs and LLMs
• - Where is potential for an integration of both approaches and what would be the benefit of such synergies for various applications?
• - Explain some ways KGs could be or already have been used to improve on LLMs.
• - Explain some ways LLMs could be or already have been used to improve on KGs.
• - Give a more detailed review, supplemented by a short demonstration of one successful realization of one combined approach (either taken from the research community or done by yourself)
• - What are future trends in the field?

Literature:

Questions:

• Intro: Prompt engineering is an emerging discipline for developing and optimizing prompts to efficiently use language models (LLMs) to enable or assist applications and research topics in a wide variety of domains, including web engineering.
• - Shortly introduce the basics of prompting.
• - Explain and demonstrate common prompting techniques.
• - Explain and (if already possible) demonstrate more complex prompting techniques currently used or discussed.
• - Review currently available libraries and tools useful for prompt engineers.
• - Discuss some methodical issues and open challenges in prompting.
• - Give a short review and outlook on current trends and future applications of advanced prompting and shortly discuss its expected impact on the field of web engineering.

Literature:

Questions:

• Intro: Prompt engineering is an emerging discipline for developing and optimizing prompts to efficiently use language models (LLMs) to enable or assist applications and research topics in a wide variety of domains, including web engineering.
• - Shortly introduce the basics of prompting.
• - Explain and demonstrate common prompting techniques.
• - Explain and (if already possible) demonstrate more complex prompting techniques currently used or discussed.
• - Review currently available libraries and tools useful for prompt engineers.
• - Discuss some methodical issues and open challenges in prompting.
• - Give a short review and outlook on current trends and future applications of advanced prompting and shortly discuss its expected impact on the field of web engineering.

Literature:

Questions:

• Intro: As a data engineer and web-developer it is inevitable to understand the concepts of data privacy and licensing and its significance in today’s data- and software-driven world.
• - Introduction and basic definitions: data, processing,
• - Personal Data: definition, types of personal data, processing principles, anonymization and pseudo-anonymization, mixed data, documentation, data subjects’ rights
• - Introduction and basic definitions: licenses
• - Open Sources Licenses: definition, types of Open Source Licenses, overview of the most popular examples of Open Source Licenses with the respective conditions, compatibility of licenses (for instance in context of data fusion), review available websites and tools for Open Source License Management, including demonstrations
• Optional: - Discuss some legal challenges that arise for training data usage in connection with AI-models
• Optional: - Name and shortly explain current or planned legislation for AI applications

Literature:

Questions:

• Provide an overview on the current state of reasearch in the automatic analysis and evaluation of web and mobile user interfaces and interactions.
• Group the existing approaches, differentiating based on their input (Textual/Source Code, Visual/Screenshots/Sketches) and purpose (page segmentation, design search/retrieval, interaction capture, reverse engineering of existing interfaces).
• How have these approaches been validated/how strong is the existing evidence of their utility? Which datasets are available?
• Which approaches are particularly benefiting from recent advancements in Deep Learning and Transformers?

Literature:

• VIPS: Deng Cai, Shipeng Yu, Ji-Rong Wen, and Wei-Ying Ma. 2003. Extracting Content Structure for Web Pages Based on Visual Representation. In Proceedings of the 5th Asia-Pacific Web Conference on Web Technologies and Applications (Xian, China) (APWeb’03). Springer-Verlag, Berlin, Heidelberg, 406–417. https://doi.org/10.1007/3-540-36901-5_42
• Michael Cormier, Karyn Moffatt, Robin Cohen, and Richard Mann. 2016. Purely Vision-based Segmentation of Web Pages for Assistive Technology. Comput. Vis. Image Underst. 148, C (July 2016), 46–66. https://doi.org/10.1016/j.cviu. 2016.02.007
• Webzeitgeist: Ranjitha Kumar, Arvind Satyanarayan, Cesar Torres, Maxine Lim, Salman Ahmad, Scott R. Klemmer, and Jerry O. Talton. 2013. Webzeitgeist: Design Mining the Web. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Paris, France) (CHI ’13). ACM, New York, NY, USA, 3083–3092. https://doi.org/10.1145/2470654.2466420
• Swire: Forrest Huang, John F. Canny, and Jeffrey Nichols. 2019. Swire: Sketch-Based User Interface Retrieval. Association for Computing Machinery, New York, NY, USA, 1–10. https://doi.org/10.1145/3290605.3300334
• Gallery D.C.: Chunyang Chen, Sidong Feng, Zhenchang Xing, Linda Liu, Shengdong Zhao, and Jinshui Wang. 2019. Gallery D.C.: Design Search and Knowledge Discovery through Auto-Created GUI Component Gallery. Proc. ACM Hum.-Comput. Interact. 3, CSCW, Article 180 (Nov. 2019), 22 pages. https://doi.org/10.1145/3359282
• WebReplay, LogRocket, mouseflow
• VINS: Sara Bunian, Kai Li, Chaima Jemmali, Casper Harteveld, Yun Fu, and Magy Seif Seif El-Nasr. 2021. VINS: Visual Search for Mobile User Interface Design. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 423, 14 pages. https://doi.org/10.1145/3411764.3445762
• Tam The Nguyen, Phong Minh Vu, Hung Viet Pham, and Tung Thanh Nguyen. 2018. Deep Learning UI Design Patterns of Mobile Apps. In Proceedings of the 40th International Conference on Software Engineering: New Ideas and Emerging Results (Gothenburg, Sweden) (ICSE-NIER ’18). Association for Computing Machinery, New York, NY, USA, 65–68. https://doi.org/10.1145/3183399.3183422

Questions:

• Provide an overview on the current state of reasearch in the automatic generation of web and mobile user interfaces.
• Particularly look for approaches which take advantage of recent advancements of Transformer models / generative AI.
• Group the existing approaches, differentiating based on their input (Textual/Source Code, Visual/Screenshots/Sketches) and purpose (Graphical Sketching, Prototyping in dedicated tools, Prototyping actual interfaces).
• How have these approaches been validated/how strong is the existing evidence of their utility? Which datasets are available? How mature is the output generated by them?

Literature:

• Pix2Code:Tony Beltramelli. 2018. Pix2code: Generating Code from a Graphical User Interface Screenshot. In Proceedings of the ACM SIGCHI Symposium on Engineering Interactive Computing Systems (Paris, France) (EICS ’18). Association for Computing Machinery, New York, NY, USA, Article 3, 6 pages. https://doi.org/10.1145/3220134.3220135
• Chunyang Chen, Ting Su, Guozhu Meng, Zhenchang Xing, and Yang Liu. 2018. From UI Design Image to GUI Skeleton: A Neural Machine Translator to Bootstrap Mobile GUI Implementation. In Proceedings of the 40th International Conference on Software Engineering (Gothenburg, Sweden) (ICSE ’18). Association for Computing Machinery, New York, NY, USA, 665–676. https://doi.org/10.1145/3180155.3180240
• Kevin Moran, Carlos Bernal-Cárdenas, Michael Curcio, Richard Bonett, and Denys Poshyvanyk. 2020. Machine Learning-Based Prototyping of Graphical User Interfaces for Mobile Apps. IEEE Transactions on Software Engineering 46,2(2020),196–221. https://doi.org/10.1109/TSE.2018.2844788
• Rewire: Amanda Swearngin, Mira Dontcheva, Wilmot Li, Joel Brandt, Morgan Dixon, and Andrew J. Ko. 2018. Rewire: Interface Design Assistance from Examples. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–12. https: //doi.org/10.1145/3173574.3174078

Questions:

• How does a Systematic Literature Review work? Prepare a guideline for computer science students explaining the main aspects and include a list of relevant publications search engines/catalogues.
• What does "systematic" mean in SLR, how is it different from other literature review methods? How does it compare to a Structured Literature Review? How does it compare to a Systematic Mapping Studies? What are risks and limitations of the method?
• How are research questions represented/quantified? What does coding mean in this context?
• How are search queries constructed? Explain the technique of query expansion for generating additional queries.
• Which SLR artifacts should be provided to allow for reproducibility and replicability?
• What tools exist to support SLRs? Demonstrate a suitable tool.

Literature:

• Kitchenham, B. (2004). Procedures for Undertaking Systematic Reviews. https://www.inf.ufsc.br/~aldo.vw/kitchenham.pdf
• Kitchenham, B., Pearl Brereton, O., Budgen, D., Turner, M., Bailey, J., & Linkman, S. (2009). Systematic literature reviews in software engineering - A systematic literature review. Information and Software Technology, 51(1), 7–15.
• Brereton, P., Kitchenham, B. a., Budgen, D., Turner, M., & Khalil, M. (2007). Lessons from applying the systematic literature review process within the software engineering domain. Journal of Systems and Software, 80(4), 571–583.
• Petersen, K., Vakkalanka, S., & Kuzniarz, L. (2015). Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology, 64, 1–18.
• Díaz, O., Medina, H., & Anfurrutia, F. I. (2019). Coding-Data Portability in Systematic Literature Reviews. Proceedings of the Evaluation and Assessment on Software Engineering - EASE ’19, 178–187.
• Khadka, R., Saeidi, A. M., Idu, A., Hage, J., & Jansen, S. (2013). Legacy to SOA Evolution: A Systematic Literature Review. In A. D. Ionita, M. Litoiu, & G. Lewis (Eds.), Migrating Legacy Applications: Challenges in Service Oriented Architecture and Cloud Computing Environments (pp. 40–71). IGI Global.
• Jamshidi, P., Ahmad, A., & Pahl, C. (2013). Cloud Migration Research: A Systematic Review. IEEE Transactions on Cloud Computing, 1(2), 142–157.
• Rai, R., Sahoo, G., & Mehfuz, S. (2015). Exploring the factors influencing the cloud computing adoption: a systematic study on cloud migration. SpringerPlus, 4(1), 197.
• A. Hinderks, F. José, D. Mayo, J. Thomaschewski and M. J. Escalona, "An SLR-Tool: Search Process in Practice : A tool to conduct and manage Systematic Literature Review (SLR)," 2020 IEEE/ACM 42nd International Conference on Software Engineering: Companion Proceedings (ICSE-Companion), 2020, pp. 81-84.
• PRISMA 2020 http://www.prisma-statement.org/

Questions:

• What is empirical Software Engineering Evaluation and how can it be done?
• Why is evaluation important in the research process?
• What is the difference between a qualitative and quantitative evaluation? (When do you use which one? What are advantages and disadvantages?)
• Prepare a list of evaluation methods and tools that can be used to evaluate software. Explain them and add relevant literature for these methods.
• Demonstrate one quantitative and one qualitative method. For this, find a feasible research question, conduct a survey on it with each of the two methods, compute the results, discuss and present them. You can choose a low level topic on your own that is related to Web Engineering. Use statistical methods to compute the results.

Literature:

• Own research
• Creswell, J. W. (2014). Research design : Qualitative, quantitative, and mixed methods approaches (4. ed., in). SAGE. https://katalog.bibliothek.tu-chemnitz.de/Record/0008891954
• Wohlin, C., Runeson, P., Höst, M., Ohlsson, M. C., Regnell, B., & Wesslén, A. (2012). Experimentation in Software Engineering. In Experimentation in Software Engineering (Vol. 9783642290). Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-642-29044-2
• Chatzigeorgiou, A., Chaikalis, T., Paschalidou, G., Vesyropoulos, N., Georgiadis, C. K., & Stiakakis, E. (2015). A Taxonomy of Evaluation Approaches in Software Engineering. Proceedings of the 7th Balkan Conference on Informatics Conference - BCI ’15, 1–8. https://doi.org/10.1145/2801081.2801084
• Wainer, J., Novoa Barsottini, C. G., Lacerda, D., & Magalhães de Marco, L. R. (2009). Empirical evaluation in Computer Science research published by ACM. Information and Software Technology, 51(6), 1081–1085. https://doi.org/10.1016/j.infsof.2009.01.002

Questions:

• What is design science research? What are the objectives of design science research?
• Which are activities? How research is conducted? How results are evaluated?
• In which research areas of computer science this methodology is more practical?
• Using design science research produce a viable and simplified artifact in the form of a construct, a model, a method and demonstrate activities

Literature:

• Own research
• Johannesson Perjons (2021), An Introduction to Design Science, https://link.springer.com/book/10.1007/978-3-030-78132-3

Questions:

• What guidelines and principles exist to safeguard Good Research Practice?
• How can these guidelines and principles be integrated into the research process?
• What is scientific misconduct/scientific malpractice?
• What is considered "high-quality research"? What are indicators thereof?

Literature:

• Own research
• The European Code of Conduct for Research Integrity http://www.allea.org/wp-content/uploads/2017/03/ALLEA-European-Code-of-Conduct-for-Research-Integrity-2017-1.pdf
• Guidelines for Safeguarding Good Research Practice https://www.dfg.de/download/pdf/foerderung/rechtliche_rahmenbedingungen/gute_wissenschaftliche_praxis/kodex_gwp_en.pdf
• Open Research Data and Data Management Plans https://erc.europa.eu/sites/default/files/document/file/ERC_info_document-Open_Research_Data_and_Data_Management_Plans.pdf

Questions:

• Provide an overview of the current state of using Generative AI with Large Language Models such as ChatGPT, Bard etc. as a tool to structure and write scientific texts (workshop/conference papers, journal articles, bachelor/master/phd theses).
• What are existing guidelines/regulations of publishers/universities? How does the usage of Generative AI need to be highlighted in the resulting texts?
• Outline the current discussion on their usage as a tool vs. authorship, intellectual property and quality concerns.
• How will the availability of Large Language Models impact academia in research and in education in the coming years?
• Experiment with a suitable model (e.g. ChatGPT, Bard), using it as a tool for writing different parts of a hypothetical master thesis (situation, motivation, problem analysis, research objectives/questions and scope, requirements, state of the art, solution draft, evaluation/experimentation plan). Ask your supervisor for the specific thesis task. Try different levels of prompts (thesis title only, title and a short description of your own, title and a detailed task description). Observe, which prompts you need, how you improve them iteratively, which ideas you have to provide, and the quality/completeness/suitability of the output.

Literature:

• https://www.nature.com/articles/d41586-023-00107-z
• https://www.acm.org/publications/policies/new-acm-policy-on-authorship
• https://www.frontiersin.org/guidelines/author-guidelines
• https://us.sagepub.com/en-us/nam/chatgpt-and-generative-ai-0
• https://www.jmir.org/2023/1/e51584
• https://www.pnas.org/post/update/pnas-policy-for-chatgpt-generative-ai
• https://dl.acm.org/doi/10.1145/3593013.3594067
• https://provost.harvard.edu/guidelines-using-chatgpt-and-other-generative-ai-tools-harvard
• https://ethics.berkeley.edu/privacy/appropriate-use-chatgpt-and-similar-ai-tools
• https://ctl.utexas.edu/5-things-know-about-chatgpt
• https://www.uni-goettingen.de/de/674738.html
• https://hss-opus.ub.ruhr-uni-bochum.de/opus4/frontdoor/index/index/docId/9734
• https://www.tu.berlin/bzhl/ressourcen-fuer-ihre-lehre/ressourcen-nach-themenbereichen/ki-in-der-hochschullehre
• https://www.scribbr.de/ki-tools-nutzen/chatgpt-universitaere-richtlinien/
• https://www.vkkiwa.de/ki-ressourcen/
• https://digital.uni-hohenheim.de/fileadmin/einrichtungen/digital/Generative_AI_and_ChatGPT_in_Higher_Education.pdf

Questions:

• What does it mean for AI to be trustworthy? How can trust be established within the context of AI in general?
• What approaches exist to make Generative AI trustworthy? What methods and ideas exist to build turstworthy audio, language and vision models like Stable Diffusion or multimodal systems like GPT-4?
• What are some examples of trustworthy AI? Demonstrate your findings by giving examples of trustworthy AI in comparsion with examples of untrustworthy patterns in AI across fields of your choice!

Literature:

• Own research

Questions:

• What are ELNs (electronic laboratory notebooks)? What different types exist?
• What are the goals and objectives of ELNs?
• How can interdisciplinary research data be managed, shared and reused in cross domain research in the context of ELNs without being invasive?
• What tools and frameworks exist for ELNs? How can these tools be extended to allow for interdisciplinary research data management? Demonstrate your findings by giving concrete examples and a short presentation on how to operate them!

Literature:

• Own research

Questions:

• What are the main conclusions of the paper "Sparks of Artificial General Intelligence"? How was the GPT-4 model evaluated? In which areas does GPT-4 excell, in which does it need further improvement?
• What other landmark papers and works exist in the context of language models, but also multimodal models like GPT-4?
• Are we heading towards AGI? What does the consensus on this topic seem to be within the scientific community?

Literature:

• https://arxiv.org/abs/2303.12712
• Own Research