Description
Evaluates an LLM's ability to identify and extract specific entities from ad descriptions, given predefined definitions and potential values for each entity.
Provider
OLX Group
Language
Polish
Evaluation
F1 score to evaluate the LLM's ability to extract entities from the given text. Ground truth was curated by initial labeling by a domain expert and finalized by a second expert after a thorough review.
Data Statistics
Number of Samples131
Collection PeriodApril 2024 - May 2024

Results based on 0 entries.

Last updated: Invalid Date

#
Model
Provider
Size
F1 Score
No results.

Rows per page

Page 1 of 0

Examples

Ad Description

Leadec jest międzynarodowym liderem w zakresie usług technicznych. W związku z rozwojem projektu we Wrześni poszukujemy kandydatów na stanowisko: Pracownik kontroli jakości Zakres obowiązków: sortowanie rozpórek lakierniczych zgodnie z wytycznymi instrukcji drobne pracę naprawcze re-work przeprowadzenie kontroli jakości zgodnie z wytycznymi instrukcji sporządzanie zapisów z przeprowadzonych inspekcji analiza niezgodności wyrobu oraz inicjowanie i koordynacja wdrażania związanych z nimi działań korygujących Oferujemy: zatrudnienie w oparciu o umowę o pracę miejsce wykonywania pracy: [adres usunięty] wynagrodzenie zasadnicze od 4300 zł brutto do 4600 zł brutto (w zależności od doświadczenia) nagrodę miesięczną w wysokości 430 zł brutto za brak absencji chorobowej 100 % finansowania szkoleń i kursów technicznych kierunkowych ubezpieczenie grupowe dla pracowników i ich rodzin pakiet medyczny za 1 zł + refundacja wizyt prywatnych do 500 zł kwartalnie dofinansowanie do wypoczynku i do pakietu sportowego codzienne wsparcie przełożonego i współpracowników możliwość rozwoju zawodowego i budowania własnej ścieżki kariery program rekomendacji pracowników parking dla pracowników Nasze oczekiwania: zdolności samodzielnego i logicznego myślenia, spostrzegawczość zdolności komunikacyjne oraz umiejętność pracy w zespole dyspozycyjność do pracy w systemie 3 zmianowym zgodnie z harmonogramem Mile widziane: doświadczenie zawodowe na podobnym stanowisku uprawnienia UDT do obsługi wózków widłowych WJO II Aplikuj lub skontaktuj się z nami. 1. Administratorem danych osobowych jest LEADEC Sp. z o.o., [adres usunięty] email: [email usunięty]; 2. Kontakt w sprawie ochrony danych osobowych możliwy jest za pośrednictwem poczty elektronicznej, pod adresem: [email usunięty] 3. Dane osobowe przetwarzane są w celu przeprowadzenia procesu rekrutacji na podstawie art. 6 ust. 1 lit.b ogólnego rozporządzenia o ochronie danych; 4. Odbiorcami danych będą osoby upoważnione przez administratora. 5. Dane osobowe będą przechowywane przez okres 6 miesięcy od daty złożenia aplikacji. 6. Kandydat ma prawo do żądania dostępu do treści swoich danych oraz prawo ich sprostowania, usunięcia, ograniczenia przetwarzania, prawo do przenoszenia danych, prawo wniesienia sprzeciwu; 7. Kandydat ma prawo do cofnięcia zgody w dowolnym momencie bez wpływu na zgodność z prawem przetwarzania (jeżeli przetwarzanie odbywa się na podstawie zgody), którego dokonano na podstawie zgody przed jej cofnięciem; 8. Kandydat ma prawo wniesienia skargi do organu nadzorczego, gdy uzna, iż przetwarzanie jego danych osobowych narusza przepisy RODO; 9. Podanie danych osobowych jest dobrowolne, jednakże brak podania danych osobowych będzie skutkować brakiem uczestnictwa w procesie rekrutacji.

Entity Definitions

{
  "manager": {
    "definition": "Whether the advertisement is looking for a manager position.",
    "type": "bool",
    "value": [
      true,
      false
    ]
  },
  "remoteRecruitment": {
    "definition": "Whether the recruitment process is remote/online.",
    "type": "bool",
    "value": [
      true,
      false
    ]
  },
  "type": {
    "definition": "The working time of the job.\nSelect one:\nfulltime: full-time work in which the employee works a standard number of hours defined by their employer, usually between 35 to 40 hours per week.\nparttime: part-time work involves working fewer hours than a full-time job, typically less than 35 hours per week. The exact number of hours can vary.\nhalftime: half-time work usually implies working half the number of hours of a full-time position. If a full-time job is 40 hours a week, a half-time job would be around 20 hours per week.\nseasonal: seasonal job is type of employment that is only available during specific times of the year, often corresponding to peak periods in certain industries. Examples include retail positions during the holiday season, agricultural work during harvest time, and roles in tourism and winter sports industries.\nb2b: cooperation or partnership B2B refers to transactions, operations, or partnerships between two businesses rather than between a business and individual consumers. This includes product supply relationships, service provision, and wholesale transactions where businesses collaborate to fulfill mutual needs and goals.",
    "type": "str",
    "value": [
      "b2b",
      "fulltime",
      "halftime",
      "parttime",
      "seasonal"
    ]
  },
  "agreement": {
    "definition": "The type of contract.\nSelect one or more:\npart: Employment Contract is a formal agreement between an employer and an employee outlining job conditions such as salary, working hours, and responsibilities. This type of contract is used most often for full time work.\ncontract: Contract work involves completing specific tasks for a company within a set time frame. Contractors are usually paid per project or hourly.\nzlecenie: Mandatory contract is an agreement where one party performs specific tasks for another in exchange of salary. This type of contract is often used for temporary or gig job.\nselfemployment: Self-employment or performing job as a business owner.\npractice: Practice/internship which is a temporary work period that provides practical experience in a specific field. Internships can be paid or unpaid and are usually for students or recent graduates to gain skills and experience.\nother: Other",
    "type": "list[str]",
    "value": [
      "contract",
      "other",
      "part",
      "practice",
      "selfemployment",
      "zlecenie"
    ]
  },
  "experience": {
    "definition": "Whether or not the job requires prior experience.\nSelect one:\nexp_no: Without experience\nexp_yes: Required experience",
    "type": "str",
    "value": [
      "exp_no",
      "exp_yes"
    ]
  },
  "availability": {
    "definition": "The type of needed availability for the job.\nSelect zero or more:\nshift_work: shift work\nweekends_work: work on the weekends\nflexible_work: Flexible working time\nFor standard 40-hour work-week, leave empty.",
    "type": "list[str]",
    "value": [
      "flexible_work",
      "shift_work",
      "weekends_work"
    ]
  },
  "workplace": {
    "definition": "The type of the workplace needed for the job.\nSelect one or more:\non_site: in the company headquarter \nhybrid: hybrid work\nremote_work_possible: remote work\ntraveling: with frequent business trips",
    "type": "list[str]",
    "value": [
      "hybrid",
      "on_site",
      "remote_work_possible",
      "traveling"
    ]
  },
  "uaFriendly": {
    "definition": "The type of people welcome to apply for this job.\nSelect one or more:\nua_people_welcome: if the job is welcoming people from Ukraine (include by default unless mentioned otherwise).\nwithout_polish: if being Polish is optional for this job",
    "type": "list[str]",
    "value": [
      "ua_people_welcome",
      "without_polish"
    ]
  },
  "specialRequirements": {
    "definition": "The special requirements for the job.\nSelect zero or more:\nsanitary_card: sanitary or health card\ndisability_certificate: Certificate of disability\nstudent_status: Student status",
    "type": "list[str]",
    "value": [
      "disability_certificate",
      "sanitary_card",
      "student_status"
    ]
  }
}

Expected Entities

{
  "jobProperties": {
    "manager": false,
    "remoteRecruitment": false,
    "type": [
      "fulltime"
    ],
    "agreement": "fulltime",
    "experience": "exp_no",
    "availability": [
      "shift_work"
    ],
    "workplace": [
      "on_site"
    ],
    "uaFriendly": [ua_people_welcome],
    "specialRequirements": null
  }
}

Ad Description

Zakład Wyrobów Metalowych TERMETAL - Jesteśmy wiodącym producentem krat pomostowych w Polsce oraz na rynku europejskim już od ponad 35 lat, jako producent mamy ustabilizowaną pozycję rynkową. Obecnie zwiększamy moce produkcyjne, dlatego poszukujemy nowych pracowników (ślusarzy, operatorów maszyn, spawaczy). ŚLUSARZ - PRACOWNIK PRODUKCYJNY praca dwuzmianowa 5500 - 9000 brutto. Miejsce pracy: Krąg (Starogard Gdański). Praca na stanowisku polega na obsłudze maszyn i urządzeń do cięcia krat stalowych, obsługa półautomatycznych linii produkcyjnych (produkcja seryjna). WYMAGANIA: Solidność i zaangażowanie w podejściu do pracy. Mile widziana umiejętność spawania MAG/MIG. OFERUJEMY: Zatrudnienie w stabilnej firmie w oparciu o umowę o pracę na pełen etat. Możliwość przystąpienia do grupowego ubezpieczenia na życie na preferencyjnych warunkach, prywatna opieka medyczna, jasne kryteria dotyczące możliwości rozwoju i awansu, jasne kryteria wynagrodzenia ustalone o wewnętrzny regulamin. Praca dwuzmianowa. Bogaty pakiet socjalny, w tym premie świąteczne oraz dopłaty do wypoczynku letniego. Bardzo dobra atmosfera pracy. Możliwość rozwoju zawodowego, w ramach prowadzonej produkcji. Zapraszamy do aplikowania tylko osoby zainteresowane długotrwałą współpracą i prosimy przekazywać swoje CV za pośrednictwem PORTALU OLX lub bezpośrednio w siedzibie firmy: ZWM TERMETAL, Krąg. DODATKOWE INFORMACJE - KONTAKT TELEFONICZNY w dni robocze pn-pt w godz. 7.00-15.00.

Entity Definitions

{
  "manager": {
    "definition": "Whether the advertisement is looking for a manager position.",
    "type": "bool",
    "value": [
      true,
      false
    ]
  },
  "remoteRecruitment": {
    "definition": "Whether the recruitment process is remote/online.",
    "type": "bool",
    "value": [
      true,
      false
    ]
  },
  "type": {
    "definition": "The working time of the job.\nSelect one:\nfulltime: full-time work in which the employee works a standard number of hours defined by their employer, usually between 35 to 40 hours per week.\nparttime: part-time work involves working fewer hours than a full-time job, typically less than 35 hours per week. The exact number of hours can vary.\nhalftime: half-time work usually implies working half the number of hours of a full-time position. If a full-time job is 40 hours a week, a half-time job would be around 20 hours per week.\nseasonal: seasonal job is type of employment that is only available during specific times of the year, often corresponding to peak periods in certain industries. Examples include retail positions during the holiday season, agricultural work during harvest time, and roles in tourism and winter sports industries.\nb2b: cooperation or partnership B2B refers to transactions, operations, or partnerships between two businesses rather than between a business and individual consumers. This includes product supply relationships, service provision, and wholesale transactions where businesses collaborate to fulfill mutual needs and goals.",
    "type": "str",
    "value": [
      "b2b",
      "fulltime",
      "halftime",
      "parttime",
      "seasonal"
    ]
  },
  "agreement": {
    "definition": "The type of contract.\nSelect one or more:\npart: Employment Contract is a formal agreement between an employer and an employee outlining job conditions such as salary, working hours, and responsibilities. This type of contract is used most often for full time work.\ncontract: Contract work involves completing specific tasks for a company within a set time frame. Contractors are usually paid per project or hourly.\nzlecenie: Mandatory contract is an agreement where one party performs specific tasks for another in exchange of salary. This type of contract is often used for temporary or gig job.\nselfemployment: Self-employment or performing job as a business owner.\npractice: Practice/internship which is a temporary work period that provides practical experience in a specific field. Internships can be paid or unpaid and are usually for students or recent graduates to gain skills and experience.\nother: Other",
    "type": "list[str]",
    "value": [
      "contract",
      "other",
      "part",
      "practice",
      "selfemployment",
      "zlecenie"
    ]
  },
  "experience": {
    "definition": "Whether or not the job requires prior experience.\nSelect one:\nexp_no: Without experience\nexp_yes: Required experience",
    "type": "str",
    "value": [
      "exp_no",
      "exp_yes"
    ]
  },
  "availability": {
    "definition": "The type of needed availability for the job.\nSelect zero or more:\nshift_work: shift work\nweekends_work: work on the weekends\nflexible_work: Flexible working time\nFor standard 40-hour work-week, leave empty.",
    "type": "list[str]",
    "value": [
      "flexible_work",
      "shift_work",
      "weekends_work"
    ]
  },
  "workplace": {
    "definition": "The type of the workplace needed for the job.\nSelect one or more:\non_site: in the company headquarter \nhybrid: hybrid work\nremote_work_possible: remote work\ntraveling: with frequent business trips",
    "type": "list[str]",
    "value": [
      "hybrid",
      "on_site",
      "remote_work_possible",
      "traveling"
    ]
  },
  "uaFriendly": {
    "definition": "The type of people welcome to apply for this job.\nSelect one or more:\nua_people_welcome: if the job is welcoming people from Ukraine (include by default unless mentioned otherwise).\nwithout_polish: if being Polish is optional for this job",
    "type": "list[str]",
    "value": [
      "ua_people_welcome",
      "without_polish"
    ]
  },
  "specialRequirements": {
    "definition": "The special requirements for the job.\nSelect one or more:\nsanitary_card: sanitary or health card\ndisability_certificate: Certificate of disability\nstudent_status: Student status",
    "type": "list[str]",
    "value": [
      "disability_certificate",
      "sanitary_card",
      "student_status"
    ]
  }
}

Expected Entities

{
  "jobProperties": {
    "manager": false,
    "remoteRecruitment": false,
    "type": [
      "fulltime"
    ],
    "agreement": "fulltime",
    "experience": "exp_no",
    "availability": [
      "shift_work"
    ],
    "workplace": [
      "on_site"
    ],
    "uaFriendly": [ua_people_welcome],
    "specialRequirements": null
  }
}

Have a unique use-case you’d like to test?

We want to evaluate how LLMs perform on your specific, real world task. You might discover that a small, open-source model delivers the performance you need at a better cost than proprietary models. We can also add custom filters, enhancing your insights into LLM capabilities. Each time a new model is released, we'll provide you with updated performance results.

Leaderboard

An open-source model beating GPT-4 Turbo on our interactive leaderboard.

Don’t worry, we’ll never spam you.

Please, briefly describe your use case and motivation. We’ll get back to you with details on how we can add your benchmark.