Post
2724
Keep track of the latest 3D releases with this space👉
dylanebert/research-tracker
- 2 replies
David Vivancos
DavidVivancos
AI & ML interests
Machine Learning, Deep Learning, NeuroTechnologies
Organizations
DavidVivancos's activity
Reacted to
dylanebert's
post with 👀
2 months ago
posted
an
update
4 months ago
Post
1676
#ICLM 2024 is almost there 🔥🔥🔥 PM if you will be in Vienna next week, Glad to catchup with the Hugging Face community there!
I would like to contribute 🎁 by releasing the sixth Knowledge Vault, with 100 lectures visualized from the last 10 years of ICML from 2014 to 2024, (10 from 2024 will be included after the conference) including knowledge graphs for all the Invited Lectures and some extras, with almost 3000 topics represented using AI.
You can explore it here:
🌏 https://theendofknowledge.com/Vaults/6/ICML-2015-2024.html
And you can learn more about the Vaults here:
📝https://www.linkedin.com/pulse/knowledge-vaults-david-vivancos-lbjef/
And previous Vaults relevant to the #huggingface community are:
🌏 [ @lexfridman 2018-2024 Interviews] https://theendofknowledge.com/Vaults/1/Lex100-2024.html
🌏 [ICLR 2014-2023] https://theendofknowledge.com/Vaults/2/ICLR2014-2023.html
🌏 [AIForGood 2017-2024] https://theendofknowledge.com/Vaults/4/AIForGood2017-2024.html
🌏 [CVPR 2015-2024] https://theendofknowledge.com/Vaults/5/CVPR-2015-2024.html
Hope you like them!
And great to see you all at #icml2024 @clem @thomwolf @julien-c and team
I would like to contribute 🎁 by releasing the sixth Knowledge Vault, with 100 lectures visualized from the last 10 years of ICML from 2014 to 2024, (10 from 2024 will be included after the conference) including knowledge graphs for all the Invited Lectures and some extras, with almost 3000 topics represented using AI.
You can explore it here:
🌏 https://theendofknowledge.com/Vaults/6/ICML-2015-2024.html
And you can learn more about the Vaults here:
📝https://www.linkedin.com/pulse/knowledge-vaults-david-vivancos-lbjef/
And previous Vaults relevant to the #huggingface community are:
🌏 [ @lexfridman 2018-2024 Interviews] https://theendofknowledge.com/Vaults/1/Lex100-2024.html
🌏 [ICLR 2014-2023] https://theendofknowledge.com/Vaults/2/ICLR2014-2023.html
🌏 [AIForGood 2017-2024] https://theendofknowledge.com/Vaults/4/AIForGood2017-2024.html
🌏 [CVPR 2015-2024] https://theendofknowledge.com/Vaults/5/CVPR-2015-2024.html
Hope you like them!
And great to see you all at #icml2024 @clem @thomwolf @julien-c and team
posted
an
update
7 months ago
Post
1945
#ICLR 2024 is almost there 🔥🔥🔥 counting the days to be again in the beautiful city of Vienna participating in the The Twelfth International Conference on Learning Representations, hope to see many of the Hugging Face comunity there!
I would like to contribute 🎁 by releasing the second Knowledge Vault, with 100 lectures visualized from the last 10 years of ICLR from 2014 to 2023, including knowledge graphs for all the Invited Lectures and some extras, with almost 3000 topics represented. (Of course using several AI tools including Llama3)
You can explore it here:
🌏 https://theendofknowledge.com/Vaults/2/ICLR2014-2023.html
And you can learn more about the Vaults here:
📝https://www.linkedin.com/pulse/knowledge-vaults-david-vivancos-lbjef/
Hope you like the Knowledge Vault!
I would like to contribute 🎁 by releasing the second Knowledge Vault, with 100 lectures visualized from the last 10 years of ICLR from 2014 to 2023, including knowledge graphs for all the Invited Lectures and some extras, with almost 3000 topics represented. (Of course using several AI tools including Llama3)
You can explore it here:
🌏 https://theendofknowledge.com/Vaults/2/ICLR2014-2023.html
And you can learn more about the Vaults here:
📝https://www.linkedin.com/pulse/knowledge-vaults-david-vivancos-lbjef/
Hope you like the Knowledge Vault!
Reacted to
akhaliq's
post with ❤️
9 months ago
Post
Design2Code
How Far Are We From Automating Front-End Engineering?
Design2Code: How Far Are We From Automating Front-End Engineering? (2403.03163)
Generative AI has made rapid advancements in recent years, achieving unprecedented capabilities in multimodal understanding and code generation. This can enable a new paradigm of front-end development, in which multimodal LLMs might directly convert visual designs into code implementations. In this work, we formalize this as a Design2Code task and conduct comprehensive benchmarking. Specifically, we manually curate a benchmark of 484 diverse real-world webpages as test cases and develop a set of automatic evaluation metrics to assess how well current multimodal LLMs can generate the code implementations that directly render into the given reference webpages, given the screenshots as input. We also complement automatic metrics with comprehensive human evaluations. We develop a suite of multimodal prompting methods and show their effectiveness on GPT-4V and Gemini Pro Vision. We further finetune an open-source Design2Code-18B model that successfully matches the performance of Gemini Pro Vision. Both human evaluation and automatic metrics show that GPT-4V performs the best on this task compared to other models. Moreover, annotators think GPT-4V generated webpages can replace the original reference webpages in 49% of cases in terms of visual appearance and content; and perhaps surprisingly, in 64% of cases GPT-4V generated webpages are considered better than the original reference webpages. Our fine-grained break-down metrics indicate that open-source models mostly lag in recalling visual elements from the input webpages and in generating correct layout designs, while aspects like text content and coloring can be drastically improved with proper finetuning.
How Far Are We From Automating Front-End Engineering?
Design2Code: How Far Are We From Automating Front-End Engineering? (2403.03163)
Generative AI has made rapid advancements in recent years, achieving unprecedented capabilities in multimodal understanding and code generation. This can enable a new paradigm of front-end development, in which multimodal LLMs might directly convert visual designs into code implementations. In this work, we formalize this as a Design2Code task and conduct comprehensive benchmarking. Specifically, we manually curate a benchmark of 484 diverse real-world webpages as test cases and develop a set of automatic evaluation metrics to assess how well current multimodal LLMs can generate the code implementations that directly render into the given reference webpages, given the screenshots as input. We also complement automatic metrics with comprehensive human evaluations. We develop a suite of multimodal prompting methods and show their effectiveness on GPT-4V and Gemini Pro Vision. We further finetune an open-source Design2Code-18B model that successfully matches the performance of Gemini Pro Vision. Both human evaluation and automatic metrics show that GPT-4V performs the best on this task compared to other models. Moreover, annotators think GPT-4V generated webpages can replace the original reference webpages in 49% of cases in terms of visual appearance and content; and perhaps surprisingly, in 64% of cases GPT-4V generated webpages are considered better than the original reference webpages. Our fine-grained break-down metrics indicate that open-source models mostly lag in recalling visual elements from the input webpages and in generating correct layout designs, while aspects like text content and coloring can be drastically improved with proper finetuning.
-
- 4 replies
Interesting to test, just downloaded the model, still shows 0 downloads... https://huggingface.co/SALT-NLP/Design2Code-18B-v0
replied to
their
post
10 months ago
Hi @clefourrier will be, atm they are drawn directly from the authors reported data in their papers linkend in the leaderboard.
posted
an
update
10 months ago
Post
Are you up to a 🤗 challenge 🏆 ?
if so 👀 Check out the new MindBigData Leaderboard 🔥🔥🔥
🚀 DavidVivancos/MindBigData-Leaderboard
Decode the "source" 🧠 with the largest multimodal opendata of brain signals for Machine Learning.
Try to beat the whopping 🥇98,97% accuracy of Smita Tiwari Shivani Goel School of CSET Bennett University, India and Arpit Bhardwaj BML Munjal University decoding the multiclass Yann LeCun mnist of brain digits caputured with EMOTIV Epoc and 🥇89,62% with Insight
Or the 🥇96,18% of Dr. Nrushingh Charan Mahapatra Intel Corporation and Prof.(Dr). Prachet Bhuyan Kalinga Institute of Industrial Technology, Bhubaneswar also with the mnist of brain digits but captured with Muse® by Interaxon Inc.
Or the 🥇85% of Matthew Zhang Westlake High School and Jeremy Lu now at Purdue University decoding brain images captured from imagenet
Or be the first to break the 🧊 with the largest open dataset 2023 (8+ billion datapoints), the multimodal MindBigData2023_MNIST-8B captured with a custom 128 channels EEG that I built and with the real 70,000 MNIST digits and put your NVIDIA gpus to work.
All the datasets are open and ready at HuggingFace, dare to try?
Hope to see you all soon in the LeaderBoard
Thanks
@DavidVivancos
if so 👀 Check out the new MindBigData Leaderboard 🔥🔥🔥
🚀 DavidVivancos/MindBigData-Leaderboard
Decode the "source" 🧠 with the largest multimodal opendata of brain signals for Machine Learning.
Try to beat the whopping 🥇98,97% accuracy of Smita Tiwari Shivani Goel School of CSET Bennett University, India and Arpit Bhardwaj BML Munjal University decoding the multiclass Yann LeCun mnist of brain digits caputured with EMOTIV Epoc and 🥇89,62% with Insight
Or the 🥇96,18% of Dr. Nrushingh Charan Mahapatra Intel Corporation and Prof.(Dr). Prachet Bhuyan Kalinga Institute of Industrial Technology, Bhubaneswar also with the mnist of brain digits but captured with Muse® by Interaxon Inc.
Or the 🥇85% of Matthew Zhang Westlake High School and Jeremy Lu now at Purdue University decoding brain images captured from imagenet
Or be the first to break the 🧊 with the largest open dataset 2023 (8+ billion datapoints), the multimodal MindBigData2023_MNIST-8B captured with a custom 128 channels EEG that I built and with the real 70,000 MNIST digits and put your NVIDIA gpus to work.
All the datasets are open and ready at HuggingFace, dare to try?
Hope to see you all soon in the LeaderBoard
Thanks
@DavidVivancos
-
- 2 replies